Source-linked AI summary
Visualization in virtual reality: a systematic review
Elif Hilal Korkut, Elif Surer
TL;DR
Traditional 2D visualizations lack a common baseline for the transition to immersive VR, while research has emphasized game technologies. This paper systematically reviews VR visualization research to characterize its theories, designs, tools, and empirical studies, finding limited standard guidance, widespread game-engine use, and continued reliance on 3D versions of traditional statistical plots.
Problem
VR visualization research lacks a common baseline and sufficiently developed standard guidelines for transitioning from traditional 2D visualizations to immersive environments.
Method
The paper conducts a systematic literature review of VR visualization studies, examining theoretical foundations, evaluation and design considerations, empirical work, methods, domains, and technologies.
Results
Only a few studies create standard VR guidelines; most use game engines, which are unsuitable for critical scientific studies, while 3D bar plots and scatter plots remain common.
Takeaways & Limitations
VR visualization needs comparative studies, objective design rules, and a common baseline to support more consistent and accurate visualization decisions.
Takeaways & Limitations
The review identifies no comprehensive visualization taxonomy specialized for VR, so existing taxonomies serve only as a base for future construction.
Abstract
from arXiv · showhide
Rapidly growing virtual reality (VR) technologies and techniques have gained importance over the past few years, and academics and practitioners have been searching for efficient visualizations in VR. To date, emphasis has been on the employment of game technologies. Despite the growing interest and discussion, visualization studies have lacked a common baseline in the transition period of 2D visualizations to immersive ones. To this end, the presented study aims to provide a systematic literature review that explains the state-of-the-art research and future trends on visualization in virtual reality. The research framework is grounded in empirical and theoretical works of visualization. We characterize the reviewed literature based on three dimensions: (a) Connection with visualization background and theory, (b) Evaluation and design considerations for virtual reality visualization, and (c) Empirical studies. The results from this systematic review suggest that: (1) There are only a few studies that focus on creating standard guidelines for virtual reality, and each study individually provides a framework or employs previous studies on traditional 2D visualizations; (2) With the myriad of advantages provided for visualization and virtual reality, most of the studies prefer to use game engines; (3) Although game engines are extensively used, they are not convenient for critical scientific studies; and (4) 3D versions of traditional statistical visualization techniques, such as bar plots and scatter plots, are still commonly used in the data visualization context. This systematic review attempts to add to the literature a clear picture of the emerging contexts, different elements, and their interdependencies.
1 Introduction
VR visualization emerges as a response to limitations of traditional 2D representations, but the field still needs theoretical grounding and consistent design guidance. The literature spans diverse immersive environments, interaction techniques, and application domains.
- Immersive visualization combines 3D models, graphs, plots, simulations, and multiple 2D representations across diverse data sources and domains.
- Traditional 2D representations are considered insufficient for some visualization needs, motivating interest in immersive VR environments.
- VR systems support varied interaction methods, including head, eye, and motion tracking, while advanced devices can provide 6 DoF.
- VR visualization requires a distinct visual language and continued theoretical development beyond the field’s strong connection with games and game technologies.
- Existing surveys often address specific domains or visualization structures, leaving a need for an overview of common problems and methods across domains.
2 Background concepts: Visual Analytics and Immersive Analytics
Visual analytics addresses analytical reasoning through interactive visual interfaces, while immersive analytics extends this agenda into collaborative and spatially oriented environments. Recent work develops techniques, frameworks, tools, and evaluation approaches for these settings.
- Visual analytics is defined as analytical reasoning facilitated by interactive visual interfaces and is positioned as a technology for handling large information volumes.
- Healthcare applications include clinical decision support, interpretable machine learning, comparative patient-record analysis, and systems such as OutFlow, CarePre, and EventAction.
- Immersive and spatial technologies create opportunities to explore complex datasets collaboratively and interactively, increasing relevance for human-computer interaction research.
- Recent research covers visualization, interaction, collaboration, perception and system evaluation, frameworks, tools, and challenge categorization.
3 Materials and Methods
The study uses a systematic literature review to examine VR visualization types, theories, research gaps, techniques, and preferred software and hardware. Studies were searched, filtered, critically appraised, and grouped by contribution, domain, and visualization category.
- The review addresses five questions covering preferred VR visualizations, research methodologies and theories, gaps, approaches and techniques, and software and hardware choices.
- Researchers followed Kitchenham and Charters’ systematic-review guidelines and searched seven academic databases for research published since 2015.
- Candidate publications were screened using inclusion and exclusion criteria, assessed by titles and abstracts, read fully, and critically appraised.
- Validated studies were grouped according to their association with visualization subfields, contribution, domain, and visualization category.
- The review separated theoretical or framework contributions from implementation-focused studies, while combination studies could appear in both result sections.
4 Results and Analysis
The reviewed literature spans visualization tools, grammars, frameworks, and VR development platforms. The field has moved from predominantly 2D toolkits toward immersive systems, including Unity-based environments and scientific visualization frameworks.
- 4.1 Tools, Toolkits, and Frameworks: Visualization tools include standalone systems, web-based presentations, software libraries, and programming-language modules, categorized by properties such as structure, scalability, and extensibility.
- 4.1 Tools, Toolkits, and Frameworks: Declarative visualization grammars range from low-level systems such as D3, Vega, and Protovis to higher-level systems such as Vega-Lite and ECharts.
- 4.1 Tools, Toolkits, and Frameworks: Traditional information-visualization toolkits largely used 2D representations, whereas scientific visualization helped drive VR systems through frameworks such as VTK and OpenVR.
- 4.1 Tools, Toolkits, and Frameworks: Unity has become a standard platform for immersive environments, supporting toolkits such as IATK and DXR for immersive data visualization.
4.2 Data Visualization
Data visualization converts information into graphical forms, but direct 3D conversion can reduce clarity through occlusion and perspective distortion. Effective VR visualization therefore combines interactivity, multiple techniques, and perspective changes to support exploration and interpretation.
- Data visualization transforms data into compact graphical information that helps audiences identify patterns, extract insights, and communicate efficiently.
- Direct conversion of 2D visualizations into 3D can cause occlusion and perspective distortion, leading to incorrect analytical interpretations.
- VR visualization should support exploratory analysis, confirmatory analysis, and structured presentation of hidden data features.
- Combining multiple visualization techniques in one VR visualization improves information flow and creates more engaging experiences.
- Switching perspectives in VR enables users to interpret data through embodied cognition and may provide more immersive experiences and precise insights.
4.3 Information Visualization
Information visualization explores effective spatial mappings and interactive computer-graphics techniques for abstract data. VR applications extend this work across cultural heritage, geographic systems, art, architecture, and computer science.
- Information visualization is an interdisciplinary field focused on spatial mappings and interactive exploration of abstract data.
- Art, Heritage, and Architecture: VR supports cultural-heritage digitization through 3D artifact and structure representations, including optimized models and texture-mapping techniques.
- Art, Heritage, and Architecture: Virtual museums can disseminate digital objects without a physical location while requiring careful choices about interaction, environment, content, and experience design.
- Art, Heritage, and Architecture: VRGIS combines VR and GIS to support spatial data query, processing, storage, and analysis, while graph partitioning can reduce urban-network clutter.
- Computer Sciences: Computer-science visualizations use methods such as GradCAM, game-engine integration, and metaphors to clarify algorithms, neural networks, and complex concepts.
4.4 Scientific Visualization
Scientific visualization uses VR to represent and explore complex, multidimensional data that conventional 2D views may not adequately explain. Applications span meteorology, geoscience, planetary science, nanoscience, materials science, and medicine, while accuracy and computational demands remain important constraints.
- VR enables interaction with 3D scientific data and can improve students’ comprehension and public engagement compared with limited 2D representations.
- High-dimensional and abstract scientific datasets can be difficult to compute and may require specialized visualizations beyond conventional desktop views.
- Meteorology and Earth Sciences: VR scientific applications include atmospheric, meteorological, geoscientific, and planetary data explored through multidimensional views, manipulation, filtering, and editing.
- Meteorology and Earth Sciences: Students and academics in extensive user tests generally agreed that VR was useful while emphasizing data and experience accessibility.
- Scientific VR representations can use artistic rendering, direct data visualization, or simplified 3D models, with optimized extraction methods helping reduce volumetric data size.
- Medicine and Biology: Medical visualization pipelines reconstruct 3D models from 2D slices or segmentation and add features such as labels, highlights, colors, selective views, and navigators.
- Medicine and Biology: Game engines are widely used for graphics performance, physics, and deployment, but their speed-oriented design can make them unreliable for accuracy-critical scientific visualization.
4.5 Collaborative VR
Collaborative VR connects users across locations to train, review, discuss, and manipulate shared data representations. Design choices concerning co-location, viewpoints, synchronization, devices, and avatars shape how collaboration is experienced.
- Collaborative virtual environments support remote interaction, training, review, and discussion through multiple information channels.
- HMDs provide faster interaction than CAVE-style facilities without major differences in accuracy or experience, supporting their preference for immersive visualization.
- Collaborative systems may be co-located or remote and may use symmetric or asymmetric interaction, with network limitations favoring co-located studies.
- Multiple viewpoints allow collaborators to use different visualizations, while shared-view designs follow a “what I see is what you see” principle.
- Collaborative tools can combine tabletop devices, HMDs, and scene-editing concepts such as containers, parallel objects, and avatars.
- Multi-user VR synchronizes separate user worlds and uses shared perspectives or customizable avatars to support presence and belonging.
4.6 Training and Simulation
VR training and simulation span professional preparation, rehabilitation, emergency response, and industrial maintenance. The literature emphasizes adaptive interaction and realism, while noting computational, cybersickness, and domain-fidelity constraints.
- VR supports training for soldiers, doctors, drivers, pilots, patient rehabilitation, and disaster management.
- Medical training can provide emergency management, cost-effectiveness, task repetition, and remote surgical training, but haptic procedures require specialized devices.
- Physics-based modeling is required to simulate deformable objects in haptic medical procedures, but realistic interaction remains computationally complex.
- Rehabilitation systems combine neuromotor hypotheses with game design and adaptive task difficulty.
- Simulation studies address emergency response by assessing situation awareness, action time, and behavior.
- High-risk industry simulators remain questionable because cybersickness, technological challenges, oversimplified environments, and insufficient realism constrain effectiveness.
4.7 Web VR
Web VR expands access to interactive and collaborative visualization but faces browser rendering and data-transfer constraints. Proposed systems address these limits through progressive loading, lightweight representations, in situ processing, and rendering optimization.
- Web services provide data access anywhere and anytime, motivating web-based visualization approaches.
- Large-scale and real-time web visualization is constrained by browser rendering capability.
- Progressive data downloading and lightweight virtual people help support online real-time fire training.
- In situ visualization processes data as simulations generate it, avoiding storage resources and allowing users to analyze immediate effects.
- VRSRAPID combines X3D virtual reality models with real-time simulations for collaborative nuclear-system scientific computing.
- A-Frame and external-data libraries support interactive web visualizations, including health-data prototypes and software-city exploration.
- Web visualization systems use texture mapping, level of detail, occlusion culling, and frustum culling to address rendering demands.
4.8 Games, Visualization and VR
Games, visualization, and VR increasingly intersect through immersive interaction, gameplay analytics, visual realism, and gamification. The literature shows useful engagement and visualization techniques but identifies limited guidance for VR gameplay data.
- VR games require new visualization techniques because immersion and embodied interaction differ from traditional video games.
- Higher geometric realism induces stronger sensations of presence and emotional responses in physiological and self-report results.
- Gameplay-data visualizations commonly use charts, diagrams, heat maps, movement visualizations, self-organizing maps, and node-link representations.
- Gameplay data can be collected through observation, questionnaires, interviews, or automatic media, producing different forms of player insight.
- Concrete visualization guidance for gameplay data in VR remains insufficient, because traditional-game taxonomies do not directly address game-specific needs.
- Gamification elements such as leaderboards, points, customizable roles, and visual appliances increase user engagement.
- VR case studies combine game-design concepts with data worlds for orbital and earth-data visualization.
- Abstract, nonintrusive visualizations in exergames use colors, shapes, and metaphors to support awareness and encourage physical activity.
4.9 Design Considerations and User Interactions
VR visualization design must account for perceptual, hardware, navigation, and interaction challenges. Studies show that locomotion interfaces and individual differences affect spatial cognition, while interactive tools support exploration of complex data.
- Evaluation is needed because visualization practice often relies on assumptions that may not be empirically correct.
- 4.9.1 Visual Perception: VR perception research must address depth, distance, shape, size, color, contrast, and hardware-related challenges.
- 4.9.1 Visual Perception: VR affordance experiments found that participants’ estimated critical angle for upright posture was lower than results from real environments.
- 4.9.2 Movement: Limited physical space motivates locomotion techniques that replace direct mapping of users’ physical movements.
- 4.9.2 Movement: Teleportation overcomes spatial constraints but can cause confusion and discontinuous feedback during movement.
- 4.9.2 Movement: Interface design and individual differences produce diverse spatial-cognition outcomes and alter users’ awareness of location.
- Immersive interaction can make experiences more effortless or cumbersome while enabling users to explore different aspects of visualized data.
- An immersive bubble chart supports exploration of unstructured data through semantic grouping, grabbing, zooming, removal, merging, and view tracking.
4.10 Comparative Studies
Comparative studies evaluate immersive visualizations against physical or alternative presentation and guidance methods, revealing trade-offs in response time, experience quality, understanding, accuracy, and workload.
- Comparative criteria: Immersive visualization comparisons examine whether virtual representations meet the threshold for choosing them over traditional alternatives.The threshold is linked to user experience and technology acceptance.
- Representation comparisons: Physicalization can decrease response time, whereas VR lag can slow participants and reduce experience quality.
- Visualization design: Multiple views and viewpoints enhanced understanding across case-study visualizations, although each technique introduces distinct optimization problems.Immersive visualizations require several parameters to be optimized simultaneously.
- Guidance comparisons: Annotations were more helpful than tutors for accuracy and task performance in a three-task guidance comparison.
5 Discussion
The discussion identifies recurring design, interaction, hardware, data, and user-centered challenges in immersive visualization. It also highlights the need for comparative evidence, flexible tools, and tested guidelines.
- Representation and interaction: Visualization studies commonly use 3D versions of bar, line, and scatter plots, while abstract visualizations support manipulation and analysis.Literal and realistic visualizations are more representative, whereas abstract forms are more open to interaction.
- Interaction and taxonomies: VR expands interaction techniques through eye gaze, head pose, walking, controllers, and teleportation, requiring broader taxonomies.
- User-centered design: Comparative studies are needed to identify interaction and visualization patterns across user groups with different capabilities, technology use, and situations.
- Guidelines and tools: Tested guidelines could support tools that create more accurate visualizations and establish a common language and visual consistency.
- Tools and data: Existing tools are often limited to particular data and visualization types, especially quantitative analysis, making qualitative toolkit development more difficult.
- Hardware and platforms: Mobile VR is unsuitable for complex visualizations because performance depends on smartphones, while browser applications face rendering and speed limitations.
- Hardware and experience: Hardware problems including cybersickness, tracking latency, and low refresh rates can disturb users and break presence, especially for complex visualizations and simulations.
- Data challenges: Data quality, stream handling, semantic extraction, multivariate compression, and feature extraction remain unresolved challenges, with automated results sometimes misinterpreted.
6 Conclusion
The review synthesizes immersive-visualization research across domains and finds limited theoretical grounding, widespread use of game technologies, persistent reliance on traditional 3D plots, and unresolved design questions.
- Review findings: The systematic review found growing immersive-visualization research across diverse domains, but only a minority of studies built a theoretical background.
- Research areas: Training, architecture, and game technologies are the most mature visualization research areas, with games technologies and computer graphics widely used.
- Research challenges: The field has generated new research questions involving usability, interactivity, and reliability.
- Standards and representations: Only a few studies create standard VR guidelines, while 3D bar plots and scatter plots remain common for data visualization.
- Tools and scientific use: Most studies prefer game engines, but their accuracy requirements are unsuitable for critical scientific studies.
- Future work: Changing underlying technology requires repeated studies and continued development of theories behind empirical research.