Source-linked AI summary
PlanePivoting: Exploration and Optimization of Gaze-Mouse Cursor Alignment for Spatial Object Translation
Jinwook Kim, Sangmin Park, Jihyeon Lee, Sang Ho Yoon, Jeongmi Lee
TL;DR
Mouse-based 2D work and gesture-based 3D manipulation can require frequent input switching, disrupting workflow and adding fatigue and cognitive load. PlanePivoting uses gaze-mouse alignment to switch translation planes, and the study reports better efficiency and satisfaction than a 3D Gizmo with comparable placement precision.
Problem
In XR, mouse input lacks the degrees of freedom for volumetric manipulation, while ray-based interactions lack precision for small 2D targets, prompting users to switch techniques between 2D and 3D work.
Source: §1 · p. 1Method
PlanePivoting uses the spatial overlap of gaze and mouse cursors to dynamically select the translation plane while users manipulate objects with the mouse.
Source: §1 · p. 2Results
PlanePivoting improved interaction efficiency and satisfaction over the 3D Gizmo while maintaining comparable placement precision; larger gaze apertures reduced hand movement but increased selection errors.
Source: §1 · p. 2Takeaways & Limitations
The study’s optimal configuration—overlapXZ mapping with a large gaze cursor—improved efficiency and subjective satisfaction over the 3D Gizmo while maintaining comparable placement precision.
Source: §8 · p. 7The evaluation focused on translation with a single fixed-size object from a stationary viewpoint and did not compare hybrid techniques or a size-matched, gaze-independent baseline.
Source: §7 · p. 7
Abstract
from arXiv · showhide
As XR matures into a ubiquitous computing platform, the disconnect between 2D and 3D input modalities remains a critical barrier to seamless workflow. Frequent transitions between the mouse for 2D precision and hand gestures for 3D manipulation induce significant physical fatigue and cognitive load. To address this, we introduce PlanePivoting, a multimodal interaction technique that extends standard mouse input into 3D space by leveraging gaze-mouse alignment. This technique dynamically modulates the translation plane based on the spatial overlap between the gaze and mouse cursor, eliminating the need for physical input modality switching. To systematically explore the foundational design space of gaze-mouse coordination and optimize key variables, we conducted a user study comparing PlanePivoting with a standard 3D Gizmo interface across two translation mapping profiles and two gaze cursor apertures. Results demonstrate that PlanePivoting outperforms the Gizmo on efficiency metrics while maintaining comparable precision and yielding higher subjective satisfaction. This study demonstrates the potential of gaze-mouse alignment for efficient spatial manipulation between 2D and 3D environments.
1 INTRODUCTION
The paper addresses the friction of switching between mouse-based 2D precision and gesture-based 3D manipulation by introducing PlanePivoting, which uses gaze-mouse alignment to switch translation planes. The study reports improved efficiency and satisfaction over a 3D Gizmo, with comparable placement precision.
1 INTRODUCTION
- PlanePivoting dynamically switches between XY and XZ translation planes according to the spatial overlap of gaze and mouse cursors.The gaze cursor acts as a mode-switching trigger while the mouse remains the direct manipulation tool.Source: §1 · p. 2
- The study explores two mapping profiles and two gaze-cursor apertures to optimize gaze-mouse coordination and compares PlanePivoting with a standard 3D Gizmo.The Gizmo is described as a common baseline for spatial translation in desktop and XR applications.Source: §1 · p. 2
- PlanePivoting significantly improved interaction efficiency and satisfaction over the 3D Gizmo while maintaining comparable placement precision.The authors also report lower cognitive load and higher usability scores than the Gizmo.Source: §1 · p. 2
- For fine depth translation, participants predominantly preferred overlapXZ, which activates the depth plane when gaze and mouse cursors overlap.Larger gaze apertures reduced hand movement but increased selection errors.Source: §1 · p. 2
2 RELATED WORK
Prior work addresses the mismatch between planar mouse input and 3D manipulation through constrained controls, modality choice, and hybrid devices. These approaches motivate a unified interaction method that retains mouse use while supporting spatial work.
2 RELATED WORK
- Constraining movement to an axis or plane can improve precision by reducing the cognitive load of controlling multiple degrees of freedom [31].Task decomposition is one way to reduce complex 6-DoF transformations to lower-dimensional operations.Source: §2 · p. 2
- In cross-reality object-transfer tasks, mouse input was fastest, while voluntary modality choice was most preferred; voluntary switching still involves cognitive load and physical effort.Mouse familiarity was associated with faster performance, while modality choice let users assign different inputs to 2D and 3D subtasks.Source: §2 · p. 2
- HandOver and SpatialMouse combine mouse precision with 3D interaction, but require physical transitions such as hovering a hand or lifting the device.HandOver uses hand hovering for direct manipulation; SpatialMouse transitions to a spatial-controller mode when lifted.Source: §2 · p. 2
3 DESIGN OF SPATIAL TRANSLATION VIA GAZE-MOUSE CURSOR ALIGNMENT
PlanePivoting extends mouse interaction into 3D translation and uses gaze-mouse alignment to select a translation plane. Its design also aims to align input mappings with users’ spatial expectations and clarify movement and interaction state.
3.1.1 Extending Mouse Input for 3D Translation
- Extending mouse interaction into 3D is intended to avoid repeated physical transitions between mouse use and hand gestures, which can fragment workflow and increase physical effort.The proposed approach lets users manipulate 3D objects with the mouse.Source: §3.1.1 · p. 2
3.1.2 Facilitating Mental Models for Spatial Interaction
- The design aims to bridge the Gulf of Execution by making 2D mouse inputs correspond intuitively to intended 3D manipulations.It draws on the principle that input-to-display mappings should align with users’ cognitive expectations.Source: §3.1.2 · pp. 2–3
3.2 PlanePivoting Interaction Design
- Gaze-mouse overlap triggers plane selection: overlapXY activates XY on overlap and defaults to XZ, while overlapXZ uses the inverse mapping.The mouse is the direct manipulation tool and gaze is the auxiliary mode-switching trigger. A semi-transparent plane and cursor color and visibility changes provide spatial and interaction-state feedback; gaze snapping and locking use dwell and hysteresis to stabilize the interaction.Source: §3.2 · p. 3, Figure 2 · p. 3
4 USER STUDY METHODS
The user study compares two gaze-cursor apertures and two translation mappings against a 3D Gizmo baseline. Participants moved a cube to targets in a volumetric task, completed randomized technique blocks, and provided performance and subjective measures.
4 USER STUDY METHODS
- The study varied gaze-cursor aperture and mapping profile, comparing small and large apertures and overlapXY and overlapXZ against a standard 3D Gizmo baseline.The apertures were 1.65° and 3.31°; the Gizmo was selected to compare techniques while keeping the user’s hand on the mouse.Source: §4 · pp. 3–4
4.1 Task
- Participants dragged a cube from a generation point to targets at two radii, with randomized depth offsets to require volumetric translation.The task represented moving objects from a 2D display into nearby 3D space.Source: §4.1 · p. 4, Figure 3 · p. 4
4.2 Procedure
- After calibration and five minutes of training, participants completed 200 trials across five techniques and two amplitudes in randomized within-subject blocks.Each technique block included 20 repetitions per amplitude; participants confirmed placements by right-clicking, and the session lasted about 90 minutes.Source: §4.2 · p. 4
4.3 Experimental Setup and Implementation
- The study ran in Unity on a Meta Quest Pro with embedded eye tracking, using filtering to smooth gaze and hand-movement input.The reported headset configuration was 90 Hz with a 111.24° field of view and a 30 Hz eye tracker.Source: §4.3 · p. 4
4.4 Evaluation Metrics
- Evaluation covered task completion time, placement offset, mouse clicks, click error rate, hand movement, usability, workload, satisfaction, and technique preference.Figure 4 plots five performance measures with 95% confidence intervals.Source: §4.4 · p. 4, Figure 4 · p. 5
4.5 Participants
- The study recruited 20 university participants, all of whom reported prior VR experience; most were right-handed.Participants’ mean age was 26.05 years (SD = 2.58).Source: §4.5 · p. 4
5 RESULTS
Across the tested PlanePivoting variants, task completion was faster than with the Gizmo, placement precision was comparable, and click counts and errors were lower. The results also show trade-offs across hand movement, cursor size, workload, usability, and participant preference.
5 RESULTS
- Of 4,000 trials, 3,876 were analyzed after 124 were excluded for failing the target threshold; data were aggregated across translation amplitudes.Comparisons used paired tests, repeated-measures analysis for PlanePivoting factors, and nonparametric tests for subjective ratings.Source: §5 · p. 5
5.1 Task Completion Time (Fig 4 (a))
- ∆=3.003 for overlapXZ-large was the largest task-completion-time reduction from the Gizmo; all four PlanePivoting variants were significantly faster.The baseline mean was 13.894, compared with 10.981 for overlapXZ-large.Source: §5.1 · p. 5
5.2 Target Position Offset (Fig 4 (b))
- No PlanePivoting variant differed significantly from the Gizmo in target-position offset, indicating comparable placement precision across mapping profiles and cursor sizes.The Gizmo mean offset was 0.595; PlanePivoting variant means ranged from 0.570 to 0.582.Source: §5.2 · p. 5
5.3 Mouse Click Count (Fig 4 (c))
- All PlanePivoting variants required significantly fewer clicks than the Gizmo; overlapXZ-large had the largest reduction, ∆=1.833.The Gizmo mean was 5.803 clicks, versus 3.991 for overlapXZ-large.Source: §5.3 · p. 5
5.4 Click Error Rate (Fig 4 (d))
- All variants reduced empty-space click errors versus the Gizmo, while overlapXY and the small cursor each had lower error rates than their alternatives.The mapping-profile means were 2.387 for overlapXY and 3.219 for overlapXZ; cursor-size means were 1.944 for Small and 3.661 for Large.Source: §5.4 · pp. 5–6
5.5 Hand Movement (Fig 4 (e))
- Hand movement was significantly lower than the Gizmo for overlapXZ-large and both overlapXY variants, but not for overlapXZ-small.The overlapXZ-small reduction did not reach significance (p=.056); mapping profile had no main effect, while cursor size approached significance (p=.053).Source: §5.5 · p. 6
5.6 Questionnaires (Fig 5)
- Compared with the Gizmo, PlanePivoting reduced several NASA-TLX demands and increased satisfaction; overlapXZ variants also scored higher on perceived usability.Gizmo usability was significantly lower than overlapXZ-small and overlapXZ-large; all five techniques exceeded the stated minimum acceptable-usability threshold.Source: §5.6 · p. 6
5.7 Ranking
- Participants preferred overlapXZ-large most, with 12 first-place rankings; 15 ranked the Gizmo last.OverlapXZ was preferred over overlapXY, and the larger cursor was preferred over the smaller one.Source: §5.7 · p. 6
6 DISCUSSION
The discussion links overlapXZ preference to how gaze-mouse alignment expresses intent, while cursor aperture trades motor effort against selection fidelity. In the tested setup, PlanePivoting improved efficiency and reduced reported cognitive and physical demands relative to the Gizmo.
6.1.1 Mental Model Alignment in Spatial Mapping
- OverlapXZ was strongly preferred: participants associated deliberate gaze-mouse alignment with intentional depth translation, while overlapXY could conflict with habitual planar mouse movement.The discussion relates the overlapXZ mapping to stimulus-response compatibility and users’ mental models of intent and action.Source: §6.1.1 · p. 6
6.1.2 Trade-off in Gaze Cursor Aperture
- Cursor aperture trades accuracy for motor ease: the small cursor had fewer selection errors, while overlapXZ-large reduced hand movement by 3.73 cm versus the baseline and was preferred by most participants.The discussion links larger apertures to accidental depth activations and suggests aperture choice depends on task context and the relative costs of effort and error.Source: §6.1.2 · p. 6
6.2 Efficiency of Gaze-Mouse Coordination
- PlanePivoting improved evaluated metrics over the Gizmo, including up to a 3s task-time improvement and fewer clicks; the authors attribute gains to removing explicit mode-switching overhead.Gaze overlap dynamically modulates the translation plane without an explicit UI toggle, and NASA-TLX ratings showed lower physical and mental demand than the Gizmo.Source: §6.2 · pp. 6–7
7 LIMITATIONS & FUTURE WORK
The evaluation is limited to 3D translation in a controlled setup, and comparisons needed to assess PlanePivoting against other approaches remain for future work.
7 LIMITATIONS & FUTURE WORK
- The study covered 3D translation with one fixed-size object from a stationary viewpoint; smaller targets may challenge 0.5s fixation, and no panning may worsen Gizmo perspective distortion.Source: §7 · p. 7
- The evaluation compared neither hybrid interaction techniques [26] nor a size-matched, gaze-independent baseline, leaving those comparisons for future work.The Gizmo comparison may conflate gaze coordination with differences in target size and gaze-snapping tolerance.Source: §7 · p. 7
- Future work should test diverse tasks and object scales, dynamic viewpoints, precise 2D operations, and alternative inputs in complex, high-frequency cross-reality workflows.The authors identify these comparisons as necessary to assess ecological validity and when continuous coordination versus hybrid switching is beneficial.Source: §7 · p. 7
8 CONCLUSION
PlanePivoting uses gaze-mouse alignment to modulate the translation plane without physical device switching. Its optimal configuration improved efficiency and satisfaction over the Gizmo while maintaining comparable placement precision.
8 CONCLUSION
- With overlapXZ mapping and a large gaze cursor, PlanePivoting improved efficiency and satisfaction over the Gizmo while maintaining comparable placement precision.Source: §8 · p. 7
- PlanePivoting uses gaze and mouse cursor alignment to dynamically modulate the translation plane, extending standard mouse input to spatial manipulation without device switching.Source: §8 · p. 7
- The strong preference for overlapXZ supports aligning interaction mechanics with users’ mental models of intent and action in spatial translation.Source: §8 · p. 7