Source-linked AI summary
Point&Spawn: Mid-Air Reference-Free Object Instantiation Using Gaze and Hand Gestures in Extended Reality
Jihyeon Lee, Ken Pfeuffer, Jinwook Kim, Jeongmi Lee
TL;DR
Reference-free mid-air spawning requires specifying a 3D position without surfaces or existing objects to guide placement. Point&Spawn stages that specification before instantiation and compares six controller-free techniques. Relative depth methods outperformed Ray Intersection, while direction modality and spawn depth showed distinct trade-offs.
Problem
In reference-free mid-air XR, users must define a new object’s position without surfaces or existing objects, making 3D position specification difficult, particularly along depth.
Source: §1 · p. 1, §2.2 · p. 2Method
Point&Spawn uses Direction Setting, Depth Setting, and Position Refinement in a continuous gesture flow, evaluated across six combinations of Gaze or NDH direction setting and three depth methods.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
The findings support prioritizing relative depth control and retaining pre-instantiation refinement within staged reference-free spawning.
Source: §8 · p. 9The study used a controlled single-object task and did not compare against an instantiate-then-reposition workflow.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Mid-air object instantiation in XR requires users to specify a 3D position without spatial references, such as surfaces or existing objects. We present Point&Spawn, a staged pipeline for pre-instantiation position specification through Direction Setting, Depth Setting, and Position Refinement within a continuous gesture flow. We evaluated six controller-free techniques combining Gaze or Non-Dominant Hand (NDH) direction setting with Ray Intersection, Relative Gain, or Drag&Hold depth setting in a user study (N=24) across Near and Far spawn depths. Relative Gain and Drag&Hold yielded faster and more accurate spawning, lower workload, higher usability, and greater preference than Ray Intersection. The shoulder-referenced NDH ray improved speed and coarse accuracy, whereas the viewpoint-based Gaze ray reduced hand movement with comparable final accuracy. Farther spawn depth imposed greater temporal costs as well as Gaze and accuracy costs with Ray Intersection. These findings offer empirical guidance for designing direction and depth control in spawning in XR.
1 INTRODUCTION
The introduction frames reference-free mid-air spawning as a problem of specifying a new object’s position in empty 3D space. Point&Spawn stages that specification before instantiation and evaluates six controller-free techniques.
1 INTRODUCTION
- Existing selection and manipulation techniques use references such as surfaces or objects, but spawning may require defining a location in empty mid-air.Systems often create objects at a default location and require subsequent repositioning, adding interaction particularly for distant placements.Source: §1 · p. 1
- Point&Spawn combines Direction Setting, Depth Setting, and Position Refinement in a continuous gesture flow before object creation.The six techniques combine Gaze or NDH direction setting with Ray Intersection, Relative Gain, or Drag&Hold depth setting.Source: §1 · pp. 1–2
- Relative Gain and Drag&Hold generally outperformed Ray Intersection in spawning time, accuracy, workload, usability, and preference.The study also found trade-offs between direction modalities and greater depth-related costs, especially for Gaze and Ray Intersection.Source: §1 · p. 2
- The shoulder-referenced NDH ray supported faster spawning and more accurate coarse placement, while Gaze reduced hand movement with comparable final accuracy.Farther spawn depth increased temporal costs more with Gaze and accuracy costs with Ray Intersection.Source: §1 · p. 2
2 RELATED WORK
Related work describes staged placement and methods for specifying 3D points, while identifying reference-free spawning as a distinct, underexplored setting.
2.1 Object Spawning in 3D XR Environment
- Spawning requires both targeting a coordinate and committing a persistent object, so efficacy depends on the broader workflow, including previews, refinement, and confirmation.Conventional staged placement can increase interaction time and effort, especially for repeated distant placements.Source: §2.1 · p. 2
- Environmental constraints, layout generation, and viewpoint stabilization can improve placement, but often rely on surfaces, scene understanding, or handheld-interface conditions.Those dependencies limit applicability to open mid-air placement without external anchors.Source: §2.1 · p. 2
2.2 3D Point Specification for Mid-air Interaction
- In 3D space, distant targets and absent environmental anchors make intended positions harder to resolve from directional cues alone.Small tremors or posture changes can also create large positional errors for ray-intersection methods at greater distances or precision demands.Source: §2.2 · p. 2
- Relative depth control broadens the controllable range, but fine-grained refinement under amplified movement remains challenging.The tradeoff between absolute position specification and relative depth control remains underexplored in reference-free object spawning.Source: §2.2 · p. 2
3 DESIGN OF OBJECT SPAWNING TECHNIQUE
Point&Spawn organizes reference-free spawning into Direction Setting, Depth Setting, and Position Refinement, linked by release, semi-pinch, and pinch gestures. The design separates direction and depth control while allowing correction before creation.
3.1 Stage 1: Direction Setting
- Gaze uses a viewpoint ray for direction, whereas NDH uses a shoulder-referenced ray that can be released from its posture once depth setting begins.The Gaze ray offers attention-driven direction setting but may be affected by involuntary eye movements and fluctuations.Source: §3.1 · p. 3, §3.1.1 · p. 3, §3.1.2 · p. 3
3.2 Stage 2: Depth Setting
- A DH semi-pinch begins depth setting along the locked direction; releasing it cancels depth setting and returns to direction specification.The blue preview cube indicates the provisional spawn location.Source: §3.2 · p. 3
- The six techniques pair Gaze or NDH direction setting with Ray Intersection, Relative Gain, or Drag&Hold depth setting.Ray Intersection uses absolute specification; the two relative methods adjust depth without aligning the DH ray to the locked direction ray.Source: Figure 2 · p. 3, §3.2 · p. 3
3.2.1 Ray Intersection (Baseline)
- Ray Intersection sets depth by aiming a DH ray along the locked direction ray and validating the closest point when the rays are within 0.2 m.The method is familiar, but may become unstable or physically demanding for far targets.Source: §3.2.1 · p. 3
3.2.2 Relative Gain
- Relative Gain maps forward or backward DH motion to incremental depth updates, with faster movement producing larger changes.The relative mapping covers a wider depth range with limited hand motion while retaining fine-grained control.Source: §3.2.2 · p. 3
3.2.3 Drag&Hold
- Drag&Hold starts continuous movement along the direction ray after DH displacement exceeds 3 cm; larger displacement increases speed, and moving back stops it.The displacement threshold acts as a dead zone against unintended activation.Source: §3.2.3 · pp. 3–4
3.3 Stage 3: Position Refinement
- After depth setting, a DH pinch starts refinement; users adjust position with 1:1 hand movement, and release commits and instantiates the object.This final correction stage allows users to compensate for earlier errors before creation.Source: §3.3 · p. 4
4 EVALUATION
The evaluation used a controlled, within-subject study to compare six spawning techniques across two target depths. It measured performance and physical movement alongside subjective workload, usability, fatigue, and preference.
4 EVALUATION
- The study asked how direction modality, depth-setting method, and spawn depth affect performance and user experience.The questions compare Gaze with NDH, absolute Ray Intersection with relative methods, and Near with Far depth conditions.Source: §4 · p. 4
4.1 Study Design
- Participants completed a within-subject 2 × 3 × 2 design: two direction methods, three depth methods, and Near and Far spawn depths.They spawned a cube at five target positions at each depth, for ten positions total; an orange marker indicated the target without serving as an environmental anchor.Source: §4.1 · p. 4
- Each trial randomly selected a target, and participants were instructed to spawn at it as quickly and accurately as possible.The study used gaze initialization and prevented the same target position from appearing in consecutive trials.Source: §4.1 · p. 4
4.2 Procedure
- Participants practiced until proficient, then completed six counterbalanced technique blocks with 40 trials per technique and post-block questionnaires.The session lasted approximately 90 minutes and included breaks and a final preference ranking and interview.Source: §4.2 · p. 4
4.3 Apparatus
- The study used a Meta Quest Pro with built-in eye and hand tracking and a desktop computer running the Unity environment.A 1C filter was applied to gaze and hand movement inputs to reduce noise.Source: §4.3 · p. 4
4.4.1 Objective Measures
- Objective measures covered direction, depth, coarse-spawn, and refinement times; coarse and final offsets; and cumulative hand movement.Coarse Offset measures accuracy after coarse spawning, while Final Offset measures accuracy after refinement.Source: §4.4.1 · pp. 4–5
4.4.2 Subjective Measures
- Subjective measures included NASA-TLX workload, SUS usability, separate hand and eye fatigue ratings, and preference rankings.Subjective ratings were collected by technique, not separately by spawn depth.Source: §4.4.2 · p. 5
4.5 Statistical Analysis
- The analysis used repeated-measures ANOVA or Aligned Rank Transform ANOVA according to distribution, with corrected post-hoc tests and separate Friedman analysis for preference.The study included 24 participants, all right-handed, with mean age 25.17 years.Source: §4.5 · p. 5, §4.6 · p. 5
5 RESULT
Across the six techniques, results show that direction modality, depth method, and spawn depth relate differently to timing at each stage.
5 RESULT
- Of 5760 collected trials, 38 (0.66%) were excluded for coarse spawn times under 1.0 second, likely reflecting accidental pinch inputs.Values are reported as mean ± standard deviation unless otherwise noted.Source: §5 · p. 5
5.1.1 Direction Setting Time
- NDH direction setting was faster than Gaze (1.46± .73 vs. 2.02 ± .78), and Far took longer than Near (1.88 ± .91 vs. 1.60 ± .66).The Gaze–NDH difference was significant for Ray Intersection and Relative Gain, but not Drag&Hold; Far took longer only within Gaze.Source: §5.1.1 · p. 5
5.1.2 Depth Setting Time
- Ray Intersection depth setting took 4.73±1.32 seconds, slower than Relative Gain (3.28±1.08) and Drag&Hold (3.08±1.39); Far took longer than Near.NDH was faster than Gaze overall, while the two relative methods did not differ significantly.Source: §5.1.2 · p. 5
5.1.3 Refinement Time
- Relative Gain refinement was fastest at 1.13 ± .36 seconds, compared with 1.30 ± .45 for Drag&Hold and 1.39 ± .47 for Ray Intersection.Far trials took longer than Near trials; the difference between depths was significant only for Ray Intersection.Source: §5.1.3 · p. 5
5.1.4 Coarse Spawn Time
- Ray Intersection coarse spawning took 6.52 ± 1.91 seconds, slower than Relative Gain (5.04±1.39) and Drag&Hold (4.85±1.54); Far took longer than Near.The Far–Near increase occurred with both Gaze and NDH and was larger with Gaze; the relative methods did not differ significantly within either modality.Source: §5.1.4 · pp. 5–6
5.1.5 Total Spawn Time
- Ray Intersection took 8.13 ± 2.12 seconds overall, versus 6.38±1.56 for Relative Gain and 6.38±1.72 for Drag&Hold; Far trials took longer than Near trials.NDH was faster overall than Gaze (6.41±1.56 vs. 7.52 ± 2.21). Far trials increased total time for both modalities, with a larger increase for Gaze.Source: §5.1.5 · p. 6
5.2 Spawning Accuracy (Figure 4 (g-i))
Both coarse and final offsets varied by depth-setting method and spawn depth, with Relative Gain yielding the smallest offsets.
5.2 Spawning Accuracy (Figure 4 (g-i))
- Offset measures are Euclidean distances from the target to the coarse or final spawned position, with lower values indicating better accuracy.Table 1 summarizes omnibus tests for objective measures.Source: §5.2 · p. 6, Table 1 · p. 7
5.2.1 Coarse Offset
- Coarse offset was .200±.141 with Relative Gain, .298±.138 with Drag&Hold, and .467±.269 with Ray Intersection; Far offsets exceeded Near offsets.NDH had a smaller coarse offset than Gaze (.305 ± .203 vs. .338 ± .238). Ray Intersection showed a larger Far–Near difference than either relative method.Source: §5.2.1 · p. 6
5.2.2 Final Offset
- Final offset was .157±.139 with Relative Gain, .236±.140 with Drag&Hold, and .395±.276 with Ray Intersection; Far offsets exceeded Near offsets.Direction setting modality had no significant main effect on final offset; Ray Intersection showed a larger Far–Near difference than the two relative methods.Source: §5.2.2 · pp. 6–7
5.3 Hand Movement (Figure 4 (i))
Hand movement differed by direction-setting modality and showed method-specific depth patterns.
5.3 Hand Movement (Figure 4 (i))
- Gaze involved less hand movement than NDH (.84m ± .38 vs. 1.00m ± .40).Spawn depth had no significant main effect on hand movement.Source: §5.3 · p. 7
- NDH involved more hand movement than Gaze at both depths, with a larger modality difference in Near conditions.The modality difference was significant within both Far and Near conditions.Source: §5.3 · p. 7
- Near trials involved more hand movement than Far with Ray Intersection, while Relative Gain showed the reverse; Drag&Hold showed no depth difference.Relative Gain also involved more hand movement than Ray Intersection overall; its difference from Drag&Hold was not significant.Source: §5.3 · p. 7
5.4 User Experience (Figure 5)
User experience varied by depth-setting method across workload, usability, and fatigue measures.
5.4.1 Task Load
- Ray Intersection produced higher total workload than Drag&Hold and Relative Gain, which did not differ significantly.Ray Intersection also scored higher on mental demand, effort, and frustration than both relative methods; for physical demand, Drag&Hold scored lower than both.Source: §5.4.1 · p. 7
5.4.2 Usability
- Ray Intersection had lower perceived usability scores than Drag&Hold and Relative Gain, which did not differ significantly.Figure 5 plots average SUS usability and NASA-TLX workload across the six techniques.Source: §5.4.2 · pp. 7–8, Figure 5 · p. 8
5.4.3 Physical Fatigue
- Hand fatigue was higher with NDH than Gaze, while eye fatigue was higher with Gaze than NDH.Drag&Hold produced less hand fatigue than either other depth method; Ray Intersection produced more eye fatigue than both relative methods.Source: §5.4.3 · p. 8
5.5 Preference Ranking (Figure 6)
Preference rankings favored both relative depth-setting methods over Ray Intersection, with Gaze+Drag&Hold ranked highest overall. Participants did not significantly distinguish between modality-matched pairs using the same depth-setting method.
5.5 Preference Ranking (Figure 6)
- Gaze+Drag&Hold had the most favorable mean preference rank (2.25), followed by NDH+Drag&Hold (2.71), Gaze+Relative Gain (2.92), and NDH+Relative Gain (3.21).NDH+Ray Intersection (4.75) and Gaze+Ray Intersection (5.17) were least preferred.Source: §5.5 · p. 8
- Preference rankings differed significantly across the six techniques (χ2(5) = 47.69, p < .001, Kendall’s W = .397).The Friedman test indicated a significant overall difference.Source: §5.5 · p. 8
- Both Ray-based techniques were ranked significantly lower than all four relative techniques (ps ≤.049).Post-hoc Nemenyi tests showed no significant difference between modality-matched pairs within the same depth-setting method (ps ≥.958).Source: §5.5 · p. 8
6 DISCUSSION
Relative Gain and Drag&Hold outperformed Ray Intersection across performance and user-experience measures, while direction modality and spawn depth shaped distinct trade-offs. The discussion translates these findings into guidance for choosing controls within the tested staged spawning design.
6.1 RQ1: Direction Setting Method Shaped Speed, Coarse Accuracy, and Physical Fatigue
- NDH produced faster spawning and smaller coarse offsets than Gaze, while the final-offset difference disappeared after refinement.Eight participants reported greater direction-setting confidence with NDH; 10 described Gaze’s single-handed operation and lower physical demand as advantages, despite its greater eye fatigue and temporal sensitivity to depth.Source: §6.1 · p. 8
6.2 RQ2: Relative Depth Setting Methods Better Supported Reference-free Mid-air Spawning
- Relative Gain and Drag&Hold yielded shorter setting and spawning times, smaller offsets, lower workload, and more favorable usability and preference than Ray Intersection.Both relative methods reduced eye fatigue; lower physical demand and hand fatigue were specific to Drag&Hold.Source: §6.2 · p. 8
6.3 RQ3: Far Spawn Depth Amplified Differences Across Direction and Depth Setting Methods
- Farther spawn depth increased time and offsets unevenly: temporal costs were greater with Gaze, while Ray Intersection showed the largest accuracy penalty.Refinement time increased with depth only for Ray Intersection; hand movement increased for Relative Gain, stayed stable for Drag&Hold, and decreased for Ray Intersection.Source: §6.3 · p. 9
6.4 Design Guidelines for Reference-free Mid-air Object Spawning
- Prefer relative depth control over intersecting rays; choose Relative Gain for accuracy and Drag&Hold for rapid traversal or lower perceived physical demand.NDH suits faster spawning, coarse accuracy, and temporal robustness across depth; Gaze suits lower hand involvement, with greater eye fatigue and temporal sensitivity to depth.Source: §6.4 · p. 9
7 LIMITATIONS AND FUTURE WORK
The study isolates core position-specification performance in a controlled, single-object spawning task. The authors identify several untested scenarios and design choices for future evaluation.
7 LIMITATIONS AND FUTURE WORK
- The controlled study used a single-object spawning task, limiting its scope to core position-specification performance.The authors call for evaluation in more realistic scenarios involving clutter or occlusion, marker-free spawning, orientation and scale adjustment, and sequential multi-object placement.Source: §7 · p. 9
- The study did not compare reference-free spawning with an instantiation-and-reposition workflow.That comparison was beyond the study’s scope and is proposed for future work in realistic placement scenarios.Source: §7 · p. 9
- Direction was locked once depth setting began, and only two spawn depths were tested.The authors note that locking direction may have reduced flexibility and call for testing continuously updated direction, additional depth levels, and sensitivity to starting depth and control settings.Source: §7 · p. 9
8 CONCLUSION
The conclusion frames reference-free spawning as staged, pre-instantiation position specification. Relative depth methods outperformed Ray Intersection, while direction modality and refinement presented distinct trade-offs.
8 CONCLUSION
- Relative Gain and Drag&Hold outperformed Ray Intersection, whose accuracy degraded most at greater depth.The depth-setting method most strongly shaped performance and user experience.Source: §8 · p. 9
- NDH improved spawning speed and coarse accuracy, whereas Gaze reduced hand involvement; refinement attenuated the coarse-accuracy difference.These findings describe a trade-off between direction-setting modalities.Source: §8 · p. 9
- The findings support prioritizing relative depth control and retaining pre-instantiation refinement within staged interaction.The work evaluates direction and depth control as part of reference-free, pre-instantiation position specification.Source: §8 · p. 9