Source-linked AI summary
Text Entry in Immersive Head-Mounted Display-based Virtual Reality using Standard Keyboards
Jens Grubert, Lukas Witzani, Eyal Ofek, Michel Pahud, Matthias Kranz, Per Ola Kristensson
TL;DR
VR applications need efficient, non-fatiguing text input despite limited visual feedback and the constraints of current head-mounted displays. This study evaluates desktop and touchscreen keyboards in VR, finding that common keyboards can transfer substantial typing performance with little learning, while keyboard repositioning has device-dependent effects.
Problem
VR applications need robust, efficient, and non-fatiguing text entry despite limited display resolution and visual feedback.
Method
The study evaluates desktop and touchscreen keyboards in VR using a 2 × 2 within-subjects experiment varying keyboard type and virtual keyboard position.
Results
Desktop keyboards retain about 60% of users’ typing speed and touchscreen keyboards about 40-45%, with little learning; repositioning preserves reasonable performance but affects devices differently.
Takeaways & Limitations
Standard keyboards can provide practical VR text entry using common hardware, and rendering keyboards and hands in front of the user can support scene-contextual placement.
Takeaways & Limitations
The evaluation focuses on users seated at a desk performing extensive text entry in minimally distracting virtual scenes, leaving engaging and dual-task scenarios for future work.
Abstract
from arXiv · showhide
We study the performance and user experience of two popular mainstream text entry devices, desktop keyboards and touchscreen keyboards, for use in Virtual Reality (VR) applications. We discuss the limitations arising from limited visual feedback, and examine the efficiency of different strategies of use. We analyze a total of 24 hours of typing data in VR from 24 participants and find that novice users are able to retain about 60% of their typing speed on a desktop keyboard and about 40-45\% of their typing speed on a touchscreen keyboard. We also find no significant learning effects, indicating that users can transfer their typing skills fast into VR. Besides investigating baseline performances, we study the position in which keyboards and hands are rendered in space. We find that this does not adversely affect performance for desktop keyboard typing and results in a performance trade-off for touchscreen keyboard typing.
1 Introduction
VR can provide an immersive, space-efficient display environment, but efficient text entry remains difficult because head-mounted displays limit visual feedback and viewing posture. This study evaluates familiar desktop and touchscreen keyboards, including repositioned virtual representations, to characterize their performance in VR.
- Motivation: Current VR text-entry methods using handheld controllers, head direction, or gaze are tedious and generally suited only to short texts.
- Motivation: Limited angular and vertical fields of view make keyboard feedback difficult to see and may require uncomfortable downward head rotation that can disrupt task context.Text and key labels require high resolution, while physical keyboards on a table are outside the HMD’s horizontal viewing position.
- Study focus: The paper studies whether familiar keyboards can support extensive, efficient, and non-fatiguing text entry in VR despite space and mobility constraints.The experiment also examines whether relocating virtual keyboards and hands affects typing performance.
- Related evidence: Prior work shows that visual feedback affects desktop-keyboard performance, while visually occluded keyboards can still yield 41.2–43.7 wpm with 8.3%–11.8% character error rates.Auto-correction reduced the reported error rates to approximately 2.6%–4.0%.
- Research gap: Desktop and touchscreen keyboards offer familiar but different trade-offs: tactile feedback may reduce visual demands, while touchscreens support reconfigurable interactions.
2 Related Work
Prior VR text-entry research spans speech, styluses, gloves, hand sensing, mobile touchscreens, head-based input, and visual feedback for physical keyboards. This work focuses on substantial text entry with familiar standard keyboards while examining how virtual hand and keyboard placement can be decoupled from physical location.
- Prior VR text entry: Earlier VR systems explored speech, tracked styluses, gloves, head-mounted touchscreens, head pointing, and gesture recognition, but these approaches involve distinct speed, learning, mobility, or editing constraints.
- Touchscreen input: Touchscreen keyboards are portable and can provide relatively high entry rates with acceptably low error rates, while hover-based VR typing has not reported an entry rate.
- Keyboard feedback: Physical desktop-keyboard studies found that removing visual feedback substantially degrades typing, whereas showing some or all of the keyboard improves performance.
- Spatial placement: Prior work on visuo-motor co-location found subtle or statistically inconclusive effects in tracked-stylus and three-dimensional spatial tasks.
- Study contribution: This paper extends the literature by testing substantial text entry on commonly available standard keyboards that require little, if any, learning in VR.
3 Repositioning Experiment
The experiment evaluates standard desktop and touchscreen keyboards in VR to establish their performance envelope and examine whether repositioning virtual keyboards and hands changes typing performance.
- Experiment aim: The experiment measures performance for familiar standard desktop and touchscreen keyboard layouts in VR, with primary emphasis on their achievable performance envelope.
- Experiment aim: It also tests whether relocating virtual keyboards and hands relative to their physical positions affects text-entry performance.
3.1 Method
The study uses a 2 × 2 within-subjects design crossing keyboard type with the virtual position of the keyboard and hands. NoReposition aligns virtual and physical locations, whereas Reposition moves the virtual representation away from them.
- Design: The 2 × 2 within-subjects design varies KeyboardType between desktop and touchscreen keyboards and VirtualKeyboardPosition between NoReposition and Reposition.
- Conditions: NoReposition aligns the virtual keyboard and hands with the physical devices, while Reposition changes their spatial location in VR.
3.2 Participants
The study recruited 27 participants familiar with QWERTZ desktop and touchscreen keyboard typing; 24 remained after exclusions.
- 27 participants were recruited from a university campus and had experience with QWERTZ desktop keyboard typing.
- 24 participants remained after two withdrawals and one exclusion for logging issues.
3.3 Apparatus and Materials
The apparatus tracked participants’ fingers and head-mounted display while rendering colored fingertip feedback in an Oculus Rift DK2.
- An OptiTrack Flex 13 system tracked fingertip and HMD positions with 0.2 mm mean spatial accuracy.
- An Oculus Rift DK2 served as the head-mounted display, with left and right fingertips shown as yellow and blue semi-transparent spheres.
3.4 Calibration Data Collection
Calibration established participants’ text-entry profiles and aligned tracked fingertip positions with their physical fingertips before the experiment.
- Participants profiled typing by copying prompted sentences with desktop and QWERTZ touchscreen keyboards.
- Figure 4 shows the touchscreen keyboard on the left and desktop keyboard on the right.
- Finger calibration estimated offsets between tracked 3D fingertip positions and nail-mounted markers by having participants touch buttons of decreasing size.
3.5 Procedure
Participants completed balanced desktop-keyboard and touchscreen-keyboard typing conditions, emphasizing speed and accuracy with error correction allowed.
- Each condition involved 15 minutes of typing stimulus sentences as quickly and accurately as possible, with backspace corrections permitted.
- Condition order was balanced across participants, with a five-minute break between conditions.
- Desktop keyboard users received tactile key feedback, while touchscreen keys lit green on touch and registered input on lift-off unless the finger slid away.
3.6 Results
Desktop keyboards supported substantially higher VR entry rates than touchscreen keyboards, while repositioning affected touchscreen speed but not desktop speed. Repositioning changed head posture markedly, without significant learning, error-rate, workload, nausea, or presence differences.
- Entry Rate: 59% versus 49% of profiling speed was retained with desktop and touchscreen keyboards, respectively, without repositioning; repositioning yielded 57% and 47%.Baseline profiling performance was 41.4 wpm for desktop keyboards and 23.98 wpm for touchscreen keyboards.
- Entry Rate: Touchscreen repositioning reduced entry rate by 2.8 wpm (32% relative), whereas desktop repositioning produced no significant speed difference.The KeyboardType × TextPosition interaction was not significant.
- Learning and Initiation: No significant improvement in learning was observed, and time to first keypress did not differ significantly across conditions.Mean time to first keypress ranged from 1.28 to 2.61 seconds.
- Error Rate: Character error rates remained low and showed no significant keyboard-type or repositioning differences.Desktop CER was 2.1% versus 2.4%, and touchscreen CER was 2.7% versus 3.6%, for NoReposition and Reposition.
- User Experience: Workload, nausea, and spatial presence did not differ significantly between repositioning conditions, while desktop conditions were preferred over touchscreen conditions.Median preference ratings were 2 for desktop and 3 for touchscreen keyboards.
- Head Posture: Mean head pitch shifted 20° for desktop and 30° for touchscreen keyboards after repositioning, with significant differences across conditions.NoReposition averaged −31° and −43°, versus −10° and −12° after repositioning for desktop and touchscreen keyboards, respectively.
4 Discussion
The study finds that desktop keyboards retain substantially higher VR typing performance than touchscreen keyboards, while repositioning trades touchscreen speed for potential ergonomic and contextual benefits. The discussion also bounds these findings to a stationary, minimally distracting text-entry setup and identifies several directions for future work.
- 4 Discussion: About 50% of standard-keyboard typing performance transferred to VR without automated error correction.Participants could correct input manually, while automatic correction was intentionally excluded because it may be context-dependent.
- 4.1 Touchscreen vs. Desktop Keyboard: Desktop keyboards were faster than touchscreen keyboards, while touchscreen tablets offered greater versatility and reconfigurability.The authors frame device choice as a trade-off among typing speed, versatility, and form factor.
- 4.2 Reposition of Hands and Keyboard: Repositioning may support typing in VR context and improve ergonomics by showing the keyboard and hands in the user’s view direction.The reported orientation differences were 20° for desktop keyboards and 30° for touchscreen keyboards, although NASA-TLX did not show a significant subjective difference.
- 4.2 Reposition of Hands and Keyboard: Repositioning imposed no significant desktop-keyboard cost but significantly degraded touchscreen-keyboard entry.The authors suggest that rotated finger motion and the need to lift fingers from the touchscreen may explain the touchscreen penalty.
- 4.3 Limitations and Future Work: The evaluation focused on seated, extensive text entry at a desk, leaving mobile, engaging, and dual-task VR scenarios for future study.The authors also note that their virtual environment minimized distractions and relied on high-precision stationary optical tracking.
5 Conclusions
The study evaluates desktop and touchscreen keyboards for text entry in head-mounted-display VR. With simple fingertip rendering, it transfers substantial typing performance without substantial learning and permits keyboard repositioning while maintaining reasonable performance.
- 5 Conclusions: Simple fingertip rendering transferred about 50% of desktop-keyboard typing performance to VR without substantial user learning.The study also maintained comparable touchscreen-keyboard performance without requiring substantial learning.
- 5 Conclusions: Repositioned keyboards and hands maintained reasonable performance and could place standard-keyboard input in the intended VR scene context.This creates opportunities to render the user’s hands at a keyboard or keypad location within the scene.
- 5 Conclusions: VR may become a work tool beyond physical screens, but robust text entry remains a barrier to that vision.The authors position this study as a step toward office-oriented VR interaction with data and applications.