Source-linked AI summary
GazeTune: Facilitating Precise Gaze-Driven Interactions with Cascaded Touch Input
Jina Kim, Eric J. Gonzalez, Yang Zhang, Sang Ho Yoon
TL;DR
GazeTune addresses the limited precision and motion stability of gaze-driven dragging by combining rapid gaze targeting with anchored touch refinement. It evaluates this cascaded interaction against gaze-only and gaze-pinch baselines, reporting lower error with comparable task completion time under motion-induced conditions.
Problem
Gaze is fast but imprecise for continuous dragging, and gaze+pinch can remain unstable during user motion.
Method
GazeTune uses gaze for coarse spatial anchoring and touch for local cursor refinement within a cascaded interaction sequence.
Results
Under Motion-Induced conditions, GazeTune achieved a 4.24% error rate and a 1.44° dropping error angle, with comparable task completion time.
Takeaways & Limitations
Gaze-and-touch integration supports precise, stable dragging while preserving rapid gaze targeting during locomotion.
Takeaways & Limitations
The evaluation used ordinary walking speed on a treadmill, fixed UI depth at 2 m, and a path-anchored interface.
Abstract
from arXiv · showhide
Eye gaze has become an essential input for spatial computing, but its coarse targeting and saccadic nature limit precision and complicate continuous interactions such as dragging, especially under user motion. Gaze+pinch has also become standard in XR for its convenience, yet mid-air gestures remain imprecise, fatiguing, and socially unacceptable. These limitations underscore the need for an approach that preserves the speed of gaze while enabling stable, fine control. We present GazeTune, a cascaded multimodal interaction technique combining gaze and touch to refine gaze-based selection and manipulation. Touch serves as a refinement channel within gaze pointing, allowing precise cursor and target control. Our work investigates how gaze-and-touch enhances dragging and mitigates Motion-Induced instability. In a study (N=20), we compared GazeTune against gaze-only and gaze-pinch methods in 2D dragging. Results show that GazeTune achieves significantly lower error with comparable execution time, validating its effectiveness and balanced trade-off between time and accuracy.
1 Introduction
GazeTune combines rapid gaze targeting with touch-based refinement to address imprecise, unstable dragging, especially during user motion. A study comparing GazeTune with gaze-only and gaze-pinch baselines found lower motion-induced errors without a substantial time penalty.
- Motivation: Gaze input is fast and low effort but lacks the stability and precision required for continuous operations such as dragging.Its saccadic nature and tracking noise complicate fine control, while gaze+pinch can remain imprecise, fatiguing, and socially intrusive.
- Related gap: Multimodal gaze techniques compensate for imprecision, but existing approaches may struggle with fluid fine-grained correction during physical movement.Prior gaze-assisted dragging work supported fluid modality transitions but did not explicitly address interaction while users were in motion.
- Approach: GazeTune pairs coarse gaze targeting with precise touch refinement across a single continuous sequence.The sequence sets a gaze start point, locks and refines it with touch, specifies a rough gaze endpoint, and refines the endpoint with touch.
- Evaluation: 4.24% error rate under Motion-Induced conditions was significantly lower for GazeTune than the compared baselines.GazeTune also produced a 1.44° dropping error angle, versus 1.89° for GazeTap and 2.91° for GazePinch.
- Contributions: The work contributes a cascaded gaze-and-touch mechanism, an empirical comparison with gaze-driven baselines, and design implications for mobile AR.The evaluation focuses on interaction precision and stability.
2 Related Work
Prior research combines gaze with other modalities to improve accuracy, but precise continuous manipulation during movement remains underexplored. GazeTune addresses this gap through cascaded touch refinement for stable dragging in motion.
- Research gap: Multimodal gaze systems have been studied for selection and manipulation, but continuous operations requiring sustained stable adjustments during movement remain underexplored.GazeTune targets this gap by integrating touch into a cascaded refinement workflow.
- Gaze limitations: Gaze alone remains error-prone for precise selection, including because of technical inaccuracies and unintended selections.Prior gaze-plus-touch work provides explicit control for correction.
- Alternative modalities: Head, hand, and other complementary modalities provide alternatives, but fine refinement during movement can be inconvenient or unreliable.Controllers and mice offer precision but reduce portability, while mid-air gestures can become unreliable during physical movement.
- Design opportunity: Wearable touch offers an unobtrusive, readily available channel for precise refinement in mobile contexts.Smartwatches avoid requiring external hardware or large, fatiguing body movements.
- Dragging under mobility: Dragging is especially vulnerable to motion-induced jitter and drift because control must be maintained across the start point, path, and endpoint.The paper evaluates gaze-and-touch support for stable acquisition and intentional real-time refinement under motion.
3 GazeTune System Design
GazeTune uses gaze for coarse spatial targeting and touch for anchored local refinement, with a Gaze Scope regulating transitions during pointing and dragging. Its interaction states support continuous correction while limiting clutching on a compact touch surface.
- Interaction structure: GazeTune combines gaze and touch in a continuous sequence supporting Point & Select and Drag & Drop.The design uses three gaze actions and four touch actions, with a 300 ms TouchDwell for confirmation.
- Pointing and selection: Gaze establishes approximate locations, while TouchDown creates a Cursor Lock and TouchMove refines the cursor locally.TouchDwell confirms selection once the refined cursor reaches the target, and TouchUp ends the interaction.
- Mode gating: The Gaze Scope gates transitions between local touch refinement and gaze-based repositioning.It dynamically extends between the initial anchor and refined cursor position as a pill-shaped region formed from circles and a connecting corridor.
- Dragging workflow: During dragging, touch controls local movement inside the Gaze Scope, while gaze outside the scope enables larger repositioning after fixation.A new fixation can establish a dragging lock, snapping the cursor to the gaze position before touch refinement resumes.
- Anchored refinement: Anchored refinement freezes the gaze cursor at a spatial reference point, allowing touch-driven local movement without prolonged visual attention.GazeTune defines pointing and dragging cursor locks for the two interaction phases.
- Evaluation setup: The study setup measures Selection Time, Drag & Drop Time, and Task Completion Time across stationary and motion-induced conditions.Participants use an HMD and smartwatch touchscreen while walking on a treadmill in the motion condition.
4 User Study
The study compares GazeTune with gaze-only and gaze+pinch techniques across stationary and motion-induced dragging, using controlled targets and performance measures. The implementation combines gaze selection with touch-based refinement and evaluates efficiency, accuracy, and workload under user mobility.
- Interaction Techniques: GazeTap uses gaze for pointing and continuous dragging, with touch contact gating selection and commitment.A 300 ms dwell confirms selection, gaze updates the drag position while touch is held, and TouchUp ends the operation.
- Interaction Techniques: GazePinch combines gaze pointing with a pinch gesture to trigger and maintain dragging, with hand movement driving the drag.The system records a gaze-anchored cursor offset on pinch-in and finalizes manipulation on pinch-out.
- Study Design: The experiment compares three techniques in Stationary and Motion-Induced contexts, with walking fixed at 1.2 m/s in the motion condition.Targets used three visual-angle amplitudes—10°, 30°, and 50°—within an ISO 9241-9 Fitts’-style selection-and-dragging task.
- Interaction Techniques: GazeTune locks the current gaze position on TouchDown and uses TouchMove deltas for continuous cursor refinement.Selection is confirmed when the refined cursor enters the target and satisfies the same dwell execution time as GazeTap; cursor locking requires fixation, out-of-scope gaze, and low touch speed.
- Participants and Measures: The study included 20 participants and 18 conditions spanning interaction technique, context, and target amplitude.The analysis measured Dropping Error, Task Completion Time, and Selection Time, among other task metrics.
- Results Presentation: Figure 5 presents results across context for each technique and reports the main effect of technique.The study collected 3,960 data points across participants, techniques, contexts, target amplitudes, and trials.
5 Result
Across stationary and motion-induced conditions, GazeTune generally reduced dragging errors and release dispersion while retaining competitive completion times, though gaze-only interaction was often faster.
- Dropping Error: GazePinch's error angle increased from 1.44° to 2.91° under Motion-Induced, while GazeTune increased from 0.92° to 1.14°.
- Dropping Error: GazeTune minimized error across amplitudes in Stationary and again had the lowest error in Motion-Induced.
- Task Completion Time: GazeTap was fastest overall, while GazeTune showed no reliable task-completion-time change between Stationary and Motion-Induced conditions.
- Selection Time: GazePinch had the shortest selection times, followed by GazeTap and GazeTune, whose times were 2.19→2.01 s across contexts.
- Error Rate: 4.24% error under Motion-Induced was lowest for GazeTune, compared with 9.70% for GazeTap and 37.12% for GazePinch.
- Release Point: GazePinch produced the largest release dispersion under Motion-Induced, whereas GazeTune was consistently the smallest across amplitudes and contexts.
6 Use Case Scenario
GazeTune supports near-to-far interaction by combining gaze-based acquisition and snapping with touch-based local refinement for precise manipulation.
- Repositioning: Users acquire a target with gaze, anchor refinement, and make small touch nudges while gaze remains within the interaction scope.
- Repositioning: Looking outside the scope triggers snapping for rapid repositioning, after which users re-enter the scope to refine final placement.
- Resizing: Resizing alternates between gaze-driven scale snapping over large distances and touch refinement for fine-scale tuning.
- Range adjustment: Range adjustment anchors and refines the start point, snaps toward a distant endpoint, then refines that endpoint precisely.
- Multi-selection: Multi-selection combines local boundary refinement with gaze-driven expansion across larger regions.
7 Discussion
GazeTune balances efficiency and robustness by combining gaze acquisition with anchored touch refinement, supporting stable control during motion while reducing reliance on prolonged fixation or mid-air movement. The discussion also identifies learning, evaluation, and deployment boundaries that motivate future work.
- Balancing Efficiency and Robustness in Mobile Gaze-Driven Interaction (RQ1): GazeTune reduces dropping errors but introduces an initial selection delay from gaze anchoring, touch refinement, and a 0.3s dwell confirmation.Users can bypass refinement when high precision is unnecessary, supporting progressive engagement.
- Balancing Efficiency and Robustness in Mobile Gaze-Driven Interaction (RQ1): Under Motion-Induced instability and longer viewing amplitudes, GazeTune provides more reliable accuracy with comparable completion times.Its reduced DDT can outweigh the slight increase in initial ST when baseline methods suffer repeated errors.
- Cascaded Synergy of Gaze and Touch (RQ2): Anchoring and re-anchoring the cursor stabilizes refinement during body motion while reducing reliance on prolonged gaze fixation and sustained mid-air arm movement.The interaction maps gaze pointing’s ballistic and homing phases to touch-stabilized homing.
- Behavioral Strategies and Experience of GazeTune (RQ3): Participants adapted quickly to GazeTune despite an initial learning curve involving visual split-attention and coordination between a 3D target and a 2D wrist interface.NASA-TLX showed no significant negative impact on cognitive demand, and the initial attentional demand diminished after a few trials.
- Limitations & Future Work: The evaluation used ordinary walking speed and treadmill motion, so its visual-motion realism and coverage of faster mobility remain limited.Future work should examine real-world walking and outdoor environments with changing visual backdrops.
- Limitations & Future Work: The study fixed UI depth at 2 m and used a path-anchored interface, leaving depth-invariant performance and other anchoring conditions for future investigation.Edge-of-watch refinement could also require restarting, motivating clutching support for safer finger repositioning.
8 Conclusion
GazeTune combines gaze for coarse acquisition with touch for anchored refinement to support precise, stable gaze-driven interaction in XR. In comparative evaluation, it achieved lower error with comparable completion time, including under motion-induced conditions.
- 8 Conclusion: GazeTune offloads coarse acquisition to gaze and uses anchored touch refinement within a continuous sequence to improve precision with minimal physical effort.The design also reduces reliance on prolonged visual fixation and clutching.
- 8 Conclusion: Comparisons against gaze-only and gaze-pinch baselines showed significantly lower error and consistent dropping accuracy under Motion-Induced conditions with comparable task completion time.Touch functions as a selective trigger for precise control during selection and drag-and-drop under mobility.
A User Feedback
User feedback contrasted gaze-only speed with precision difficulties, gaze-pinch intuitiveness with arm fatigue, and GazeTune’s initial learning curve with later accuracy and lower attentional demand.
- GazeTap: GazeTap was perceived as easiest and fastest, but participants reported difficulty aligning the cursor precisely and lower confidence in final placement.Some compensated with subtle head movements or prolonged visual effort, which caused eye tiredness.
- GazePinch: GazePinch felt natural and direct in stationary use, but extended-arm posture and repeated correction attempts became tiring, especially during motion.Participants described the repeated reach-out and re-attempt cycles as mentally taxing.
- GazeTune: GazeTune initially felt less intuitive and slower than GazeTap or GazePinch, but most participants found it easy after a few trials and valued its accuracy.Users appreciated refining with a finger while maintaining softer visual focus and reducing head movement.