PaperlayerView exact revision on arXiv

Source-linked AI summary

All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda

Lik-Hang Lee, Tristan Braud, Pengyuan Zhou, Lin Wang, Dianlei Xu, Zijun Lin, Abhishek Kumar, Carlos Bermejo, Pan Hui

arXiv:2110.05352v3cs.CY

TL;DR

The metaverse lacks a comprehensive view connecting current technologies with the requirements of a unified virtual ecosystem. This survey provides a technology-and-ecosystem framework, reviews enabling technologies and user-centric factors, and proposes research challenges and opportunities toward the metaverse.

  • Problem

    Existing work does not comprehensively connect current technologies with the requirements for reaching the metaverse across technology and ecosystem dimensions.

    Source: §I · p. 3

  • Method

    The paper surveys enabling technologies and user-centric ecosystem factors, then proposes research challenges and opportunities for metaverse development.

    Source: §I · p. 3

  • Results

    The survey delivers a comprehensive technology-and-ecosystem framework and identifies research challenges and opportunities toward the ultimate stages of the metaverse.

    Source: §I · p. 3

  • Takeaways & Limitations

    The framework offers a structured path for examining and advancing the metaverse across its technological and ecosystem dimensions.

    Source: §I · p. 3

Abstract

from arXiv · show

Since the popularisation of the Internet in the 1990s, the cyberspace has kept evolving. We have created various computer-mediated virtual environments including social networks, video conferencing, virtual 3D worlds (e.g., VR Chat), augmented reality applications (e.g., Pokemon Go), and Non-Fungible Token Games (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have bought us various degrees of digital transformation. The term `metaverse' has been coined to further facilitate the digital transformation in every aspect of our physical lives. At the core of the metaverse stands the vision of an immersive Internet as a gigantic, unified, persistent, and shared realm. While the metaverse may seem futuristic, catalysed by emerging technologies such as Extended Reality, 5G, and Artificial Intelligence, the digital `big bang' of our cyberspace is not far away. This survey paper presents the first effort to offer a comprehensive framework that examines the latest metaverse development under the dimensions of state-of-the-art technologies and metaverse ecosystems, and illustrates the possibility of the digital `big bang'. First, technologies are the enablers that drive the transition from the current Internet to the metaverse. We thus examine eight enabling technologies rigorously - Extended Reality, User Interactivity (Human-Computer Interaction), Artificial Intelligence, Blockchain, Computer Vision, IoT and Robotics, Edge and Cloud computing, and Future Mobile Networks. In terms of applications, the metaverse ecosystem allows human users to live and play within a self-sustaining, persistent, and shared realm. Therefore, we discuss six user-centric factors -- Avatar, Content Creation, Virtual Economy, Social Acceptability, Security and Privacy, and Trust and Accountability. Finally, we propose a concrete research agenda for the development of the metaverse.

I. INTRODUCTION · II. RELATED WORK AND MOTIVATION

The paper defines the metaverse as a blended physical-digital environment progressing from digital twins through digital natives toward persistent, interoperable physical-virtual co-existence. It surveys this development through technological enablers and ecosystem factors because existing research remains fragmented and immature.

I. INTRODUCTION

  • The metaverse blends physical and digital environments through Internet, Web, and Extended Reality convergence, with users represented by avatars in an alternate virtual life.The concept has been described through varied formulations, including lifelogging, collective virtuality, embodied Internet, mirror worlds, and simulation venues –.Source: §I · p. 1
  • Metaverse development proceeds through digital twins, digital natives, and finally persistent co-existence of physical and virtual reality, or surreality.Digital twins duplicate physical entities and reflect properties such as motion, temperature, and function through linked data,.Source: §I · pp. 1–2, Figure 1 · p. 1
  • The final metaverse stage requires interoperable, self-sustaining virtual worlds where avatars can participate across platforms while maintaining identity, content, and experience.Content created in one game could transfer to another, while metaverse agents and avatars interact with physical devices, robots, and wearable displays.Source: §I · p. 2
  • Current cyberspace applications remain incomplete because games lack interoperability, social virtual spaces are not perpetual, and few systems achieve shared, open, experience-duality worlds.The paper places these limitations within a progression from information-rich media and personalization toward persistent physical-virtual experience-duality,.Source: §I · pp. 2–3
  • The survey proposes a technological framework, reviews enabling technologies against metaverse requirements, and identifies research challenges and opportunities across technology and ecosystem dimensions.Its stated contribution is a comprehensive view organized around technologies such as edge computing, XR, and artificial intelligence and their ecosystem implications.Source: §I · p. 3, Figure 3 · p. 3

II. RELATED WORK AND MOTIVATION

  • A 2012–2021 literature review retained 30 English research papers after searches of ACM Library and IEEE Xplore yielded 67 keyword-matched entities.The initial venue-restricted search produced only two effective studies, prompting relaxed venue criteria and exclusions of non-research formats.Source: §II · p. 4
  • Existing studies address scalability, interoperability, interaction techniques, activity analysis, and collaborative virtual environments, but these contributions remain fragmented across applications and system problems.Examples include workload balancing, 3D-model conversion, synchronized physical-virtual gestures, avatar-behavior clustering, drone interaction, and virtual learning spaces –.Source: §II · p. 4
  • Only one full survey, two minisurveys, and three position papers were found, motivating a holistic review connecting eight technological topics with six ecosystem building blocks –.The survey targets researchers and stakeholders across computing, social science, economics, content creation, policy, governance, and game-oriented development.Source: §II · pp. 4–5

III. FRAMEWORK … B. Augmented Reality (AR)

The framework organizes the metaverse into technology enablers and ecosystem factors, while the XR discussion explains how VR and AR connect users with shared virtual and physical environments. AR extends this connection into everyday surroundings through increasingly seamless interaction, tracking, and wearable access.

III. FRAMEWORK

  • The framework divides fourteen metaverse focus areas into technology pillars that enable the ecosystem and user-centric factors governing participation, ownership, social norms, security, privacy, and trust.The technology side includes XR, interaction, AI, blockchain, computer vision, robotics/IoT, edge/cloud computing, and future networks; the ecosystem side includes avatars, content creation, virtual economy, social acceptability, security/privacy, and trust/accountability.Source: §III · pp. 5–6
  • The framework positions edge-cloud synergy and advanced mobile networks as infrastructure supporting computer vision, AI, robotics, and IoT while improving performance and user experience.Edge computing handles delay-sensitive, bandwidth-hungry processing locally, whereas cloud computing provides scalable computation and storage.Source: §III · p. 6
  • The paper reviews the framework’s fourteen aspects across technologies, ecosystem actors, social norms, and a final research agenda for a unified, shared virtual space.The sequence spans XR, interaction, robotics/IoT, AI, blockchain, computer vision, edge computing, networks, avatars, content creation, virtual economy, social acceptability, privacy/security, and trust/accountability.Source: §III · p. 6

IV. EXTENDED REALITY (XR)

  • XR derives from the Reality-Virtuality Continuum and spans alternate realities that bridge virtual entities with physical environments, with the survey focusing on VR, AR, MR, and holographic technologies.The continuum includes branches leaning toward physical reality, while the paper progressively discusses four primary reality types emphasized by academia and industry.Source: §IV · p. 6

A. Virtual Reality (VR)

  • VR places users in fully synthetic environments where interaction, shared presence, real-time collaboration, and synchronized object-avatar states are essential but difficult to scale with low latency.Unlimited concurrent users acting on virtual objects can create synchronization and latency challenges that negatively affect user experience.Source: A. Virtual Reality (VR) · p. 6

B. Augmented Reality (AR)

  • AR overlays computer-generated content onto physical surroundings, evolving from visually focused, sedentary systems toward mobile and multisensory interaction that bridges users with the metaverse.Potential perceptual channels include audio, visuals, smell, and haptics, although early systems primarily delivered visual overlays through tangible controllers.Source: B. Augmented Reality (AR) · p. 7
  • Seamless AR interaction requires lightweight interfaces, spatial detection and tracking, and support for simultaneous physical and virtual actions in everyday environments.Freehand systems such as Voodoo Dolls and HOMER illustrate pinch-based and ray-casting interaction, while outdoor AR requires mapping digital content to real-world positions.Source: B. Augmented Reality (AR) · p. 7
  • AR content can reach users through headsets, handheld devices, projectors, and tabletops, but headsets preserve attention and free users’ hands for a more seamless metaverse experience.Recent headsets improve mobility and provide visual and audio cues, while smell and haptic feedback remain comparatively neglected.Source: B. Augmented Reality (AR) · p. 7

C. Mixed Reality (MR) … B. New Human Visions via Mobile Headsets

The paper positions mixed reality as a bridge between physical and virtual worlds and surveys technologies for interacting with and displaying metaverse content. It highlights interoperability, alternative mobile inputs, holographic delivery, and mobile-headset constraints such as limited field of view and information overload.

C. Mixed Reality (MR)

  • MR connects physical environments, users, and virtual entities through environmental understanding and interoperability, making it a starting point for the metaverse despite lacking a definitive definition –,,,.Six working definitions include MR as the continuum’s middle space, AR synonym, collaboration, AR–VR combination, environmental alignment, or a stronger form of AR, –.Source: C. Mixed Reality (MR) · pp. 7–8

D. Large Display, Pico-Projector, Holography

  • Large displays, pico-projectors, and smartphone-embedded projectors may deliver metaverse content publicly because mobile headsets’ social acceptability remains uncertain,.These channels can let users without mobile headsets view realistic digital entities, while smartphones support content sharing anytime and anywhere.Source: D. Large Display, Pico-Projector,… · p. 8
  • Holography could merge volumetric 3D objects with urban environments and strengthen presence, but current systems remain constrained by resolution, display size, and device mobility,.Reflection-based holography provides highly realistic colour reproduction, whereas laser-driven holography can produce skin-sensed aerial images within regions no larger than 5 cm3,.Source: D. Large Display, Pico-Projector,… · p. 8, Figure 7 · p. 8

V. USER INTERACTIVITY

  • The user-interactivity survey targets ubiquitous interaction among people, avatars, and virtual objects across physical environments, addressing the limits of keyboard-and-mouse input for body movement and mobility [32],,.It covers interaction techniques, display technologies, feedback cues, and haptic telepresence across the extended-reality continuum.Source: §V · p. 8, A. Mobile Input Techniques · p. 8

A. Mobile Input Techniques

  • Mobile input research combines computer vision with optical, inertial, infrared, electromagnetic, capacitive, on-body, and textile interfaces to support natural, subtle, and mobile interaction, –.Computer vision offers intuitive interaction but can require substantial computation and introduce latency; combining it with sensors can support both intuitive and time-sensitive manipulation.Source: A. Mobile Input Techniques · p. 9

B. New Human Visions via Mobile Headsets

  • Mobile AR/MR headsets align physical and virtual views while preserving mobility, but their typical sub-60-degree fields of view and bulky frames can reduce immersion, task performance, and peripheral awareness,,, –.Contact-lens displays may alleviate frame occlusion and provide directional overlays, although current prototypes offer low-resolution visuals.Source: B. New Human Visions via Mobile Headsets · p. 9
  • Peripheral overlays combined with audio and haptic feedback can present navigation cues while minimizing interruption, but effective display requires controlling placement, readability, visual fatigue, shakiness, and information volume,, –.Edge-of-field overlays align with walking and other locomotive activities, allowing users to attend to physical-world tasks while receiving metaverse information.Source: B. New Human Visions via Mobile Headsets · pp. 9–10, Figure 8 · p. 10

C. The importance of Feedback Cues … B. Connected vehicles

The paper presents multimodal feedback, telepresence networking, and XR-IoT integration as foundations for realistic metaverse interaction. It further applies these ideas to connected vehicles through augmented visualization, digital twins, safety support, and shared mobility experiences.

C. The importance of Feedback Cues

  • Haptic interaction requires mobile form factors such as exoskeletons, gloves, finger addenda, and wristbands, using mechanisms including air-jets, ultrasounds, and lasers –.Simulated haptic cues can compensate for missing physical feedback through virtual springs, redirected manipulation, stiffness, and object weighting –.Source: C. The importance of Feedback Cues · p. 10
  • Multimodal visual, auditory, and haptic cues improve user experience, responsiveness, task accuracy, and virtual-object acquisition, while supporting visually impaired users –.The paper calls for combining feedback modalities across varying metaverse scenarios and exploring smell and taste.Source: C. The importance of Feedback Cues · p. 10

D. Telepresence

  • 60% bandwidth reduction from Deadband compression enables cutaneous haptic feedback while managing Just Noticeable Difference (JND) for distinguishable stimuli.The approach addresses synchronization requirements for telepresence by transmitting haptic data separately.Source: D. Telepresence · pp. 10–11
  • A 1 ms Tactile Internet latency target supports real-time haptic telepresence, but device latency also matters: smartphone glass-to-glass latency reaches 19.18 ms –.The paper identifies aggregate latency, network capacity, and mobile energy and computation constraints as barriers requiring edge or cloud offloading.Source: D. Telepresence · p. 11

A. VR/AR/MR-driven human-IoT interaction

  • XR interfaces compensate for missing tangible controls on miniature IoT devices and remotely control bulky devices, while visualizing otherwise invisible IoT operations and data risks –.The section frames XR-IoT integration around data management, visualization, and human-IoT interfacing.Source: A. VR/AR/MR-driven human-IoT interaction · p. 11, §VI · p. 11
  • AR/MR supports natural-gesture IoT control, robot task authoring, collaborative drone search, and drone-path editing through virtual overlays, –[191].The interaction models vary by rendered-content scale and device or identifier controllability.Source: A. VR/AR/MR-driven human-IoT interaction · p. 11, Figure 10 · p. 12

B. Connected vehicles

  • Connected vehicles use AR/MR to explain autonomous driving, improve road safety, provide real-time navigation, and share views among vehicles –.Nissan’s I2V links physical and virtual worlds through omni-sensing, real-time road analysis, and digital twins of vehicles, people, buildings, and environments.Source: B. Connected vehicles · pp. 12–13

C. Robots with Virtual Environments … B. Computer Agent

The surveyed metaverse vision combines virtual environments, robots, artificial intelligence, digital twins, and computer agents to support interactive, adaptive, and increasingly autonomous systems. AI applications are organized around automatic digital twins, computer agents, and avatar autonomy, while robot interfaces and learning-based controllers address collaboration and behavior generation.

C. Robots with Virtual Environments

  • Virtual environments enable users to understand robot operations, collaboratively plan tasks, and build trust toward human-robot collaboration.AR/VR/MR interfaces support task-scenario and safety analysis, while research examines user perception and interface design.Source: C. Robots with Virtual Environments · p. 13
  • Robots connected through mixed-reality interfaces are being considered for collaborative work and diverse everyday roles, including companionship, care, inspection, and home security.The survey reports high acceptance of collaborative robots and natural interaction with robots and drones.Source: C. Robots with Virtual Environments · p. 13
  • Metaverse digital twins can serve as virtual testing grounds for robot designs, allowing evaluation of user perceptions before physical implementation.Digital twins can model altered living spaces, including mechanised walls that adapt spatial separation to users’ activities and social contexts.Source: C. Robots with Virtual Environments · p. 13

VII. ARTIFICIAL INTELLIGENCE

  • AI supports metaverse creation and operation by addressing automatic digital twins, computer agents, and avatar autonomy amid vast amounts of sophisticated data.The survey presents AI as a means to automate monitoring, regulating, planning, and other data-analysis tasks.Source: §VII · p. 14

A. Automatic Digital Twin

  • An automatic digital twin fuses historical metaverse and physical-system data for training, then combines real-time data for model inference using deep learning.Deep learning can automate feature extraction and data processing, supporting classification, recognition, prediction, and determination services for physical entities.Source: A. Automatic Digital Twin · p. 14
  • Digital twins support applications including remote surgery, elder monitoring, smart-city planning, environmental analysis, traffic testing, energy prediction, and industrial production optimisation.These applications connect physical and information systems or combine IoT, BIM, historical production data, and machine-learning methods.Source: A. Automatic Digital Twin · pp. 14–15

B. Computer Agent

  • Computer agents or NPCs generate non-player behavior through control strategies, animations, graphics, and voice, with finite-state methods offering simplicity but poor scalability.FSM-based behavior trees and decision trees represent states and actions, while SVM controllers have limited state, behavior, and decision-making flexibility.Source: B. Computer Agent · p. 15
  • Reinforcement learning enables NPCs to learn from interaction rewards, customize behavior to entertainment goals, and adjust skill levels as game complexity increases.Deep reinforcement learning combines neural networks with reinforcement learning to handle large state spaces, exemplified by AlphaGo’s matrix-based action selection.Source: B. Computer Agent · p. 15

C. Autonomy of Avatar · VIII. BLOCKCHAIN · A. Data storage

The surveyed metaverse architecture combines autonomous avatars with blockchain-based, decentralised data storage. Avatar research focuses on realistic creation and modelling, while blockchain addresses massive-data, security, privacy, integrity, and voting requirements.

C. Autonomy of Avatar

  • Avatars represent metaverse players and support interaction with people or computer agents, but existing games often produce highly similar avatars because customization uses limited models or sub-models,.Social applications additionally require avatars whose facial and motion characteristics reflect physical humans.Source: C. Autonomy of Avatar · p. 15
  • GAN-based methods generate 2D avatars and real-time 3D meshes and textures, while Chalas et al. develop autonomous 3D avatar generation from face scanning rather than 2D images, [271]–.GANs train generator and discriminator networks adversarially until generated images are not recognized as fake.Source: C. Autonomy of Avatar · p. 15
  • AI-driven autonomous avatars can reproduce players’ behavior when they are absent, as Forza Motorsport’s Drivatars learn driving data and generate non-deterministic virtual players with the same driving style.The learned data include road position, race line, speed, braking, and acceleration; a related neural-network framework is reported in.Source: C. Autonomy of Avatar · p. 16
  • Neural networks outperform decision trees and Naive Bayes for learning players’ shooting styles, while reinforcement learning performs much better than neural networks in fighting games,.The shooting-style experiments covered moving direction, leap moment, and accelerator; the fighting-game models used identical fighting data.Source: C. Autonomy of Avatar · p. 16

VIII. BLOCKCHAIN

  • Metaverse-scale digitisation generates vast data that cannot feasibly be uploaded to centralised clouds under limited network resources, motivating blockchain for decentralised and secure storage –.The data include digital twins, user avatars, and large-scale fine-grained maps.Source: §VIII · p. 16
  • Blockchain stores data in chronologically linked blocks replicated across nodes that synchronize through consensus, so decentralisation and peer-held records support security and error correction, [285].The section identifies Bitcoin, proposed in 2009, as the most famous blockchain application.Source: §VIII · p. 16

A. Data storage

  • Blockchain-based storage addresses centralised systems’ high capacity costs and privacy-leakage risks by allowing users to create, validate, and record data blocks cooperatively.Proposed mechanisms include certificate-less signatures to protect access keys, regenerative codes to repair tampered data, and incentive-based storage with separate data and access-control blockchains –.Source: A. Data storage · p. 16
  • Blockchain voting can prevent intentional and accidental tampering, but requires solutions for authenticating real voters and auditing results without exposing private information, [293].Anonymous Kerberos addresses voter authentication, while asymmetric and threshold encryption protect voting content and private keys during auditing.Source: A. Data storage · pp. 16–17

B. Data sharing … B. Human Pose & Eye Tracking

The surveyed metaverse infrastructure combines blockchain mechanisms for secure sharing and interoperability with computer-vision methods for spatial understanding, avatar control, and immersive interaction. Key challenges remain in adapting these techniques to real-time, resource-constrained, diverse, and increasingly complex environments.

B. Data sharing

  • Blockchain storage encrypts and relocates data to anonymous nodes, but conventional sharing remains unsupported and requires additional key-management mechanisms.Users can act as both data requesters and providers, while all blockchain nodes record data locations for convenient access.Source: B. Data sharing · p. 17
  • Key-management, proxy re-encryption, smart contracts, and access control extend blockchain storage to secure and auditable data sharing,,.The reviewed mechanisms integrate keys with metadata, protect them in untrusted settings, and track user data-access behaviour.Source: B. Data sharing · p. 17

C. Data interoperability

  • Blockchain supports secure interoperability among organizations, including shared customer data between banks and insurers for separate business operations.The motivation is to manage privacy and security when multiple parties access and operate on shared data.Source: C. Data interoperability · p. 17
  • In smart healthcare, blockchain-based medical data systems target authentication, interoperability, and confidentiality for patient digital twins accessed by multiple doctors,,.Distributed ledgers also create opportunities for aggregation and anonymisation through mining.Source: C. Data interoperability · p. 17

IX. COMPUTER VISION

  • Computer vision underpins XR by processing visual inputs to understand users and surroundings, supporting reliable virtual and augmented environments.The review covers localisation and mapping, human tracking and pose estimation, scene understanding, and image processing for degraded inputs.Source: §IX · p. 17

A. Visual Localisation and Mapping

  • Visual SLAM reconstructs unknown 3D environments and estimates device motion through feature extraction, frame-to-point-cloud mapping, and loop detection,,,.ORB-SLAM variants work well in AR, while mobile deployment remains constrained by real-time computation; future metaverse systems require greater precision and efficiency for object registration and physical-world interaction.Source: A. Visual Localisation and Mapping · p. 18

B. Human Pose & Eye Tracking

  • Human pose tracking represents avatar body posture through joint positions or body-part key points, while eye tracking enables gaze prediction and intent inference for immersive XR interaction,,,,,.Reliable eye tracking must operate in real time across diverse users, locations, and visual conditions within device power and computation limits, requiring high-quality datasets and deep learning –.Source: B. Human Pose & Eye Tracking · pp. 18–19

1) Human Pose Tracking: … 1) Semantic Segmentation and Object Detection:

Computer vision is central to metaverse perception, requiring robust tracking, eye-based depth and gaze estimation, holistic scene understanding, and real-time semantic segmentation and object detection. Key challenges include occlusion, illumination, depth ambiguity, computational cost, privacy, unseen classes, tiny objects, and diverse physical–virtual scenes.

1) Human Pose Tracking:

  • Human pose tracking must handle self-occlusion, multi-user robustness, and extreme illumination, so methods commonly combine RGB with infrared or depth sensors, –[326].OpenPose supports 2D tracking across VR, AR, and metaverse environments, while FingerTrack targets 3D finger and hand tracking; both single- and multi-person tracking remain necessary.Source: 1) Human Pose Tracking · p. 19

2) Eye Tracking:

  • Eye tracking estimates gaze and vergence using eye geometry and infrared cameras, but immersive systems still need accurate depth estimation for virtual objects.Eye tracking can reduce rendering costs by rendering only the user’s view and support physical–virtual integration, yet focus-blur deficiencies, occlusion, motion sickness, and eye fatigue remain challenges.Source: 2) Eye Tracking · p. 19

C. Holistic Scene Understanding

  • Holistic scene understanding combines semantic segmentation, object detection, stereo matching, depth estimation, and action interpretation to support interaction with physical and virtual objects and users.It addresses what surrounds the user, how far objects are, and what those objects may be doing, making it pivotal to metaverse operation.Source: C. Holistic Scene Understanding · pp. 19–20

1) Semantic Segmentation and Object Detection:

  • Semantic segmentation is crucial for metaverse scene understanding, but real-time accuracy remains challenging because AR applications require roughly 60 frames per second.Recent methods use image crop/resizing, efficient network design, and transfer learning, with some achieving real-time segmentation in MR.Source: 1) Semantic Segmentation and Object… · p. 20
  • Object detection enables XR interactions by locating and classifying faces, text, markers, and generic categories so virtual objects can be attached to and manipulated with physical ones.Although text, face, and generic-category detectors have matured for XR, metaverse deployment still requires robustness to tiny objects, varied camera fields of view, diverse data distributions, and increasing computation, motivating lightweight methods.Source: 1) Semantic Segmentation and Object… · pp. 20–21
  • Metaverse segmentation must distinguish virtual from real pixels, handle diverse objects and avatars, and adapt to complex or previously unseen classes.Pixel-wise processing also creates substantial computation and memory demands.Source: 1) Semantic Segmentation and Object… · p. 20
  • Face detection in the metaverse must recognize both physical and synthetic faces despite occlusion, pose changes, illumination variation, and stricter privacy requirements.Future research should improve robustness and establish clearer rules or criteria for face detection.Source: 1) Semantic Segmentation and Object… · p. 20

2) Stereo Depth Estimation: … X. EDGE AND CLOUD

The section surveys computer-vision capabilities needed for immersive metaverse interaction, including depth and action understanding, image restoration and enhancement, and edge computing for timely processing. It also identifies challenges involving heterogeneous real-time activity, realism, cross-world content, and mobile-device computation.

2) Stereo Depth Estimation:

  • Metaverse depth estimation must support precise placement of connected digital and real contents while handling heterogeneous avatar activities in real time, requiring sophisticated sensors and algorithms.Source: 2) Stereo Depth Estimation · p. 21

3) Action Recognition:

  • Action recognition enables avatars and XR devices to interpret human, avatar, and object behavior for interaction and context-specific feedback using cameras, body sensing, RGB data, and sensor fusion [326],, [405]–.Deep learning also shows potential for emotion recognition in VR, while effective feedback should combine local and global pose information.Source: 3) Action Recognition · p. 21, 2) Stereo Depth Estimation · p. 21
  • Future metaverse action-recognition systems should become more adaptive and robust across virtual worlds while addressing multimodal catastrophic forgetting and emotional understanding.Source: 3) Action Recognition · p. 21

D. Image Restoration and Enhancement

  • Image restoration and enhancement are needed to maintain clean, high-resolution virtual displays despite blur, noise, haze, illumination changes, rain, motion, and distant users.Restoration reconstructs clean images, whereas enhancement improves image quality; both support denoising, deblurring, and avatar super-resolution –.Source: D. Image Restoration and Enhancement · pp. 21–22

1) Image Restoration:

  • Restoration methods support VR colour and texture recovery, artefact removal, diminished reality, dehazing, and blur-aware rendering, but should better integrate real and virtual contents with avatar interaction, –.Source: 1) Image Restoration · p. 22

2) Image Enhancement:

  • Super-resolution improves XR realism, view quality, and motion-sickness-related display limitations, while future systems could jointly learn restoration, enhancement, and downstream localization, tracking, and scene-understanding tasks, [404], –.Source: 2) Image Enhancement · p. 22

X. EDGE AND CLOUD

  • Because metaverse computation is too intensive for mobile and wearable interfaces, edge offloading is proposed to address cloud latency, interaction timing, congestion, and privacy challenges [432],.The reviewed edge-computing solution targets timely processing and user experience; Figure 19 depicts AR/VR latency from edge to cloud.Source: §X · pp. 22–23, Figure 19 · p. 23

A. User Experienced Latency … D. Versus Cloud

The metaverse requires latency below human perceptual limits, motivating edge computing and MEC to support immersive, real-time interaction despite cloud and mobile-device constraints. Edge improves latency, mobility, privacy, and local orchestration, while cloud remains essential for large-scale storage and computation through coordinated hybrid architectures.

A. User Experienced Latency

  • Offloading reduces mobile computation and memory burdens but adds networking latency, creating a viewport-size tradeoff that may favor physical deployment improvements over resource orchestration alone.Longer latency requires larger locally navigable viewports and more streamed content, which can further increase latency.Source: A. User Experienced Latency · p. 23
  • Only 24 of 184 countries reliably met the MTP threshold via wired networks, and only China met it via wireless networks, showing cloud reachability limitations.MTP latency measures the time between a user action and its displayed effect; exceeding perceptual limits can cause lag, sickness, and dizziness.Source: A. User Experienced Latency · p. 23
  • 60% less latency was achieved by LTE cloudlets than default cloud offloading, while edge measurements also found at least 80 ms lower average service latency.Edge computing places computation, storage, and transmission closer to users, reducing latency relative to cloud offloading.Source: A. User Experienced Latency · p. 23

B. Multi-access edge computing

  • MEC places edge servers near cellular users, reducing packet round-trip time and enabling near-real-time orchestration for nearby multi-user interactions such as social AR.5G promises down to 1 ms last-mile latency, while MEC supports one-hop transmission for local AR content management.Source: B. Multi-access edge computing · p. 24

C. Privacy at the edge

  • Edge processing can preserve privacy by keeping wearable data local and aggregating model updates, but heterogeneous edge deployments require mutual authentication across multiple trust domains.Federated learning can train eye-tracking or motion-tracking data locally while sending parameter updates rather than private raw data.Source: C. Privacy at the edge · p. 25

D. Versus Cloud

  • Cloud remains superior for long-term, large-scale storage and economic operations because centralized datacenters offer greater capacity and reliability for the metaverse’s massive data.The paper estimates the metaverse could be 1,000 times Earth’s size in 20 years, requiring robust cloud service for large concurrent populations.Source: D. Versus Cloud · p. 25
  • A cloud-edge orchestrator should assign latency-tolerant management to the cloud and real-time processing, nearby interaction, and private data handling to the edge.Hybrid architectures can distribute local view updates and rendering to the edge while retaining global game-state updates and extensive storage in the cloud.Source: D. Versus Cloud · pp. 25–26

XI. NETWORK … C. Network-aware applications

The metaverse depends on pervasive future mobile networking to support computation, communication, and shared experiences, while demanding massive throughput and extremely low latency. Its networks must also incorporate user experience, mobility, sensing, and application-driven adaptation to coordinate transmission with changing network conditions.

XI. NETWORK

  • Metaverse applications will rely heavily on 5G and beyond to provide pervasive access for remote computation, large databases, automated systems, and shared user experiences.Source: §XI · p. 26

A. High Throughput and Low-latency

  • Metaverse multimedia traffic will require massive bandwidth, while interactive applications require latency from roughly 130 ms for gaming down to milliseconds for immersive XR and haptics.High-resolution video is expected to dominate traffic, followed by data and metadata from pervasive sensors; requirements may exceed 5G’s available bandwidth.Source: A. High Throughput and Low-latency · p. 26
  • Current 5G cannot guarantee end-to-end ultra-low latency: RAN improvements are limited, core-network communication often contributes 10–20 ms, and URLLC remains commercially undeployed.Edge computing benefits may therefore be limited unless servers connect directly to the 5G gNB; reliable low-latency service also requires coordination among developers, ISPs, and content providers.Source: A. High Throughput and Low-latency · p. 26
  • URLLC, mMTC, network slicing, and mmWave address distinct metaverse needs, but face deployment, orchestration, spectrum-sharing, range, and obstacle-penetration constraints.mMTC supports autonomous devices such as buildings, cities, robots, drones, and vehicles, while mmWave can provide up to 800 MHz channels and sub-1 ms RAN latency in dense deployments.Source: Figure 21 · p. 26, A. High Throughput and Low-latency · p. 27

B. Human- and user-centric networking

  • User-centric networking should measure and optimize Quality of Experience beyond basic latency and throughput, incorporating application, human, and contextual factors.QoE models differ across applications such as cloud gaming and mobile XR, and the metaverse must use them to proactively manage traffic and user expectations.Source: B. Human- and user-centric networking · p. 27
  • Mobility, handovers, ISP choice, location, and heterogeneous infrastructure can destabilize latency and other user-experience parameters, while redundant embodied-sensor data can consume scarce network resources.User-centric designs should account for transient handover latency and aggregate redundant information from sensors that monitor users and their environments.Source: B. Human- and user-centric networking · pp. 27–28

C. Network-aware applications

  • Network-aware applications can combine QoE and application-usage signals with exposed network conditions so applications adapt content transmission, such as reducing video resolution, to maximize user experience.5G capability exposure can inform path selection, congestion handling, and application-layer data rates across cloud and edge scenarios.Source: Figure 22 · p. 28, C. Network-aware applications · p. 28

XII. AVATAR … B. Censorship

The section surveys avatars as user representations shaping virtual interaction, then examines XR content creation, collaboration bottlenecks, and censorship challenges in the metaverse. It proposes broader avatar design considerations and comprehensive censorship metrics for increasingly interconnected virtual worlds.

XII. AVATAR

  • Avatars represent users and other embodied agents across digital worlds, with editable appearances, scanned likenesses, cartoon faces, and roles such as friends or coaches,,.The discussion focuses on digital representations while noting social robots could connect human users with virtual entities through emotion-aware interaction and telepresence [555],,.Source: §XII · p. 28
  • Avatar appearance and behavior shape realism, presence, trust, body ownership, satisfaction, and social interaction, while body sensing can improve real-time gestures and physiological expression –.Relevant design factors include facial detail, micro-expressions, body completeness, style, fidelity, gestures, movement synchronization, and multi-avatar cooperation.Source: §XII · p. 29
  • Avatar research should address in-the-wild behavior, contextual appropriateness, avatar-induced behavior, privacy, fairness, and connections with physical worlds –.The Proteus Effect and self-perception theory link avatar characteristics to behavioral and attitudinal changes, while designers should provide diverse appearances and privacy-preserving identity protection –.Source: §XII · p. 29

A. Authoring and User Collaboration

  • XR authoring systems let users create digital objects through VR interfaces, reusable patterns, smart wearables, interactive characters, AR sketches, overlays, motion capture, and physical-object integration, –.These systems span VR, AR, and MR environments and support both artistic and functional content creation.Source: A. Authoring and User Collaboration · p. 30
  • Scaling metaverse content creation requires organizing interactive stories, enabling collaboration among avatars, and supporting heterogeneous devices, with limited work exploring role-based collaboration,, –.Examples assign participants roles such as wizards, observers, facilitators, directors, actors, and cinematographers.Source: A. Authoring and User Collaboration · p. 30
  • User-generated content can support remote physical and virtual tasks, education, conferencing, social gatherings, commerce, and collaborative live-streaming through XR overlays –.The metaverse could connect speakers and viewers on a shared virtual landscape while enabling collaborative creation.Source: A. Authoring and User Collaboration · pp. 30–31

B. Censorship

  • Internet censorship uses mechanisms including IP filtering, domain blocking, keyword targeting, and manual content categorization, but probing approaches have limited scalability and often infer censored content –.Prior studies examined censorship across countries including Iran, Egypt, Sri Lanka, Norway, Pakistan, Syria, and Arab-world states.Source: B. Censorship · p. 31
  • Metaverse censorship could block virtual objects or environments, restrict avatar speech and gestures, damage interoperability and records, and motivate comprehensive metrics modeled on Censored Planet.The Uncensored Library in Minecraft illustrates a virtual refuge for censored information, while proposed metrics would measure censorship causes, severity, and prevalence.Source: B. Censorship · p. 31

C. Creator Culture … C. Metaverse commerce

The section examines creator culture and virtual-economy governance, focusing on collaborative content production, cryptocurrency stability, market concentration, and interoperability in metaverse commerce. It identifies design, regulatory, and technical conditions needed for an inclusive and functional metaverse economy.

C. Creator Culture

  • Creator culture may enable all avatars to co-create virtual entities through accessible authoring systems, novice incentives, human-AI collaboration, and token-based rewards.Alien Worlds illustrates how players’ efforts can be converted into NFTs and tangible real-world rewards.Source: C. Creator Culture · p. 32
  • Digital-content proliferation raises capacity and preservation questions, motivating virtual museums and equitable access to virtual entities and their physical affordances,,,,,.Digital culture may also influence real-world working relationships,.Source: C. Creator Culture · p. 32

XIV. VIRTUAL ECONOMY

  • The virtual economy requires economic governance spanning internal and external regulation, industry market structure, and support for user activity and content creation.Internal and external economic governance form the bedrock of the virtual economy.Source: §XIV · p. 32, Figure 25 · p. 32

A. Economic Governance

  • Cryptocurrency may face deflationary pressure and speculation, requiring trusted mechanisms such as fractional-reserve banking, rebasing, or stablecoins while preserving decentralisation concerns.No consensus exists on the severity of cryptocurrency deflation, and banking systems may conflict with decentralisation.Source: A. Economic Governance · pp. 32–33
  • Governments and international bodies should monitor metaverse competition, support trusted cryptocurrency, and address systemic and illegal-activity risks associated with financial governance.State interventions can shape currency-market sentiment, but authorities may struggle to act as lenders of last resort.Source: A. Economic Governance · p. 33

B. Oligopolistic Market

  • High development costs may produce an oligopolistic metaverse dominated by major technology firms, although differentiated products and non-collusion could limit consumer harm,.Concentration also creates substantial profit and privacy concerns because dominant firms could oversee users’ parallel-world experiences.Source: B. Oligopolistic Market · pp. 33–34, Figure 27 · p. 34

C. Metaverse commerce

  • Metaverse commerce extends digital trade beyond e-commerce by emphasizing interoperability, allowing users to carry possessions across multiple virtual worlds.Existing games generally provide only limited, unilateral, or developer-controlled transfers rather than complete interoperability.Source: C. Metaverse commerce · pp. 34–35, Figure 28 · p. 35
  • Open-metaverse interoperability requires common technical frameworks because separate developers face difficulties connecting virtual spaces and exchanging content; emerging groups propose shared protocols [57].Existing games lack natural integrated transfers and inter-game user-to-user trade, limiting user freedom.Source: C. Metaverse commerce · p. 35

D. Virtual Objects Trading … E. Cyberbullying

The metaverse requires interoperable and trustworthy systems for trading virtual objects, while its social acceptability depends on addressing privacy, diversity, fairness, addiction, and cyberbullying. These challenges require verifiable safeguards, inclusive design, accountable algorithms, and effective detection and mitigation mechanisms.

D. Virtual Objects Trading

  • Virtual-object trading can build on NFT marketplaces and auctions, but cross-world incompatibility makes interoperability and a common framework essential.Figure 29 conjectures trading between users from different virtual worlds through portals, while existing systems such as OpenSea, Rarible, Battle Pets, and My DeFi Pet provide real-world exemplars.Source: D. Virtual Objects Trading · pp. 35–36, Figure 29 · p. 36
  • Counterfeit virtual objects can undermine ownership, pricing, buyer trust, and creator incentives, so developers should combine stringent review with neural-network similarity checks.NFT fraud and insufficient verification can produce a market-of-lemons distortion, especially because virtual-object value may depend heavily on originality, rarity, and social status.Source: D. Virtual Objects Trading · p. 36

XV. SOCIAL ACCEPTABILITY

  • Social acceptability spans privacy threats, user diversity, fairness, addiction, cyberbullying, device acceptability, cross-generational design, avatar acceptance, and sustainability.The section frames these as design factors influencing whether the metaverse can be accepted by society.Source: §XV · p. 36

A. Privacy threats

  • Privacy must be designed early through verifiable protections for bystanders and user-consented privacy trading that preserves innovation while compensating users fairly.Existing bystander-privacy solutions rely on device owners’ good intentions, whereas privacy trading could support data flows and broader social acceptability.Source: A. Privacy threats · p. 37, Figure 30 · p. 37

B. User Diversity

  • Inclusive metaverse design must accommodate diverse users and content while promoting personalised yet fair recommendations that minimise biased effects on behaviour and decisions.Relevant users include children, elderly people, disabled individuals, and people across race, gender, and religion.Source: B. User Diversity · p. 37

C. Fairness

  • Because autonomous governance agents can systematically disadvantage groups, metaverse designers should make algorithmic fairness a core value through transparency, outcome controls, and community participation.User perceptions of fairness matter, but relying on perceived fairness alone can introduce outcome-favorability bias.Source: C. Fairness · pp. 37–38

D. User Addiction

  • Immersive metaverse use may produce addiction, escapism, psychological harm, and risky behavioural changes, while super-realistic environments could intensify usage and require further in-the-wild study.Prior evidence concerns social networks, mobile applications, smartphones, VR, AR, and virtual meetings; the discussion of metaverse addiction is explicitly intended to provoke research attention rather than present evidence.Source: D. User Addiction · p. 38

E. Cyberbullying

  • Cyberbullying threatens the metaverse’s viability, but algorithmic detection, fairness, and mitigation are difficult because vague misconduct becomes harder to recognise across complex 3D worlds.After identification, care, virtual social support, and self-disclosure can be deployed as mitigation solutions.Source: E. Cyberbullying · p. 38

F. Other Social Factors … B. Ethical designs

The metaverse raises unresolved social-acceptability, privacy, security, environmental, and governance challenges as immersive devices and persistent virtual worlds connect diverse users. Proposed responses include privacy-preserving avatar and world copies, ethical design, and federated or code-based governance, but worldwide regulation remains difficult.

F. Other Social Factors

  • Social acceptability remains underexplored for metaverse devices and virtual worlds, while headset safety can harm users and nearby bystanders.Limited studies address acceptability in virtual worlds, with little coverage of digital twins or the metaverse.Source: F. Other Social Factors · pp. 38–39
  • Metaverse designers must support cross-generational user cohorts and address acceptability of avatars representing deceased people in persistent virtual environments.Existing social networks serve demographics unevenly [681], while virtual immortality and Digital Humanity raise questions about family acceptance and interactions with long-lived avatars,.Source: F. Other Social Factors · p. 39
  • The metaverse’s massive computational infrastructure may increase energy consumption and pollution, making sustainability indices and green-computing design important for adoption.Environmental responsibility may affect user attitudes, active users, and opposition to the metaverse, while sustainability-based sourcing and construction are proposed,.Source: F. Other Social Factors · p. 39

XVI. PRIVACY AND SECURITY

  • Connected devices continuously collect personal, behavioral, and communication data, creating privacy risks that users may accept in exchange for perceived benefits.VR haptics and wearables that track fine-grained movement can introduce new threats, despite benefits such as GPS-based friend discovery.Source: §XVI · p. 39

A. Privacy behaviours in the metaverse

  • Metaverse users can be monitored, eavesdropped on, extorted, or surveilled because avatars reproduce real-world privacy behaviors while users remain unaware of their virtual surroundings.Attackers can monitor activities and interactions, and immersive worlds can expose users to observation across virtual and physical environments.Source: Figure 31 · p. 39, A. Privacy behaviours in the metaverse · p. 40
  • Multiple avatars, temporary private world copies, and invisible avatar copies can confuse attackers or prevent eavesdropping, but shared-resource merging creates consistency and user-experience challenges.Private copies require API-based merging of purchases and coordinated handling when multiple copies use the same resources.Source: A. Privacy behaviours in the metaverse · p. 40
  • Deep-fakes, alternate representations, dark patterns, and personalized bots can manipulate users into revealing information, making unwanted decisions, or purchasing deceptive products.Attackers may imitate friends, induce fear or urgency, exploit prior observations, and use inferred personality traits to create nudged social interactions.Source: A. Privacy behaviours in the metaverse · p. 40

B. Ethical designs

  • Ethical metaverse design requires governance spanning jurisdictions, balancing code-based monitoring and bans against the risk of interference, anarchy, or chaos.Second Life illustrates country-based regulation and developer-enforced rules through logs; proposed alternatives include federated neighborhoods with affinity rules and blockchain-supported punishments.Source: B. Ethical designs · pp. 40–41

1) Digital twins protection: … B. Informed Consent

The section frames metaverse trust, privacy, and accountability around protecting digital twins and biometric data while designing verifiable, dependable, and inclusive consent mechanisms. It highlights blockchain, trusted execution, situational awareness, preference-based consent, and stronger safeguards for minors and vulnerable users.

1) Digital twins protection:

  • Blockchain can protect original digital twins by verifying records through peer-to-peer validation and enabling creators to sell them as unique NFT assets.Proposed applications include blockchain storage of biometric health records that digital twins can use.Source: 1) Digital twins protection · p. 41

2) Biometric data:

  • Biometric sensing makes avatars and interactions more immersive, but sensitive movement, health, and activity data require protection while remaining accessible to digital twins and wearables.Gyroscopes, gloves, suits, and haptic feedback support realistic avatar control and interaction.Source: 2) Biometric data · p. 41

XVII. TRUST AND ACCOUNTABILITY

  • Metaverse adoption depends on users’ willingness to trust its systems and on accountability when unintended consequences occur.The section presents trust and accountability as conditions for the convergence of Internet, Web technologies, and XR.Source: §XVII · p. 41

A. Trust and Information

  • A verifiable trust mechanism is needed because immersive metaverse users may expose their actions to unknown parties and face misinformation, scams, and overtrust in major brands.These risks are especially consequential for vulnerable users, including older adults, while XR misuse can cause significant physiological trauma.Source: A. Trust and Information · pp. 41–42
  • Trust could be strengthened by unobtrusive situational-awareness displays and by using trusted execution environments for secure, isolated data processing and remote attestation.TEE deployment in XR remains insufficiently mature to guarantee real-time experiences.Source: A. Trust and Information · p. 42

B. Informed Consent

  • Informed consent should let avatars accept or refuse checkable information, but facial-expression limitations weaken trust cues available in face-to-face communication.A metaverse consent mechanism must account for sensitive information leaving the owner’s control.Source: B. Informed Consent · p. 42
  • Metaverse providers should strengthen verification and regularly review protections for minors, who may not understand data-processing risks and whose parental consent is difficult to establish.The Age Appropriate Design Code is identified as a relevant source of recommendations.Source: B. Informed Consent · p. 42
  • Consent mechanisms for vulnerable populations should avoid requiring harmful disclosures, while data-driven consent can adapt permissions to changing privacy preferences beyond lengthy default forms.Vulnerable users may face disproportionate privacy, safety, discrimination, or well-being harms.Source: B. Informed Consent · p. 42

C. Accountability · XVIII. RESEARCH AGENDA AND GRAND CHALLENGES

Accountability is essential to trust and the metaverse’s benefits, requiring responsibility, recourse, privacy-aware data handling, content governance, auditing, and liability frameworks. The research agenda addresses eight enabling technologies and six ecosystem dimensions for a perpetual, shared, immersive virtual-physical universe.

C. Accountability

  • Accountability requires responsibilities, incentives, and recourse for XR stakeholders, while content moderation must distinguish human avatars from auto-trolls amid cross-border free-expression conflicts.XR data practices also require minimisation, zero-knowledge retention, consented policy pipelines, and decision provenance, while auditing secondary data and assigning liability for machine-triggered physical harms remain unresolved.Source: C. Accountability · pp. 42–43

XVIII. RESEARCH AGENDA AND GRAND CHALLENGES

  • The agenda envisions perpetual, shared, concurrent, three-dimensional virtual spaces and calls for holistic development across fourteen interrelated technology and ecosystem areas.The proposed roadmap combines technology enablers with ecosystem drivers to build self-sustaining, autonomous, perpetual, unified, and endless cyberspace.Source: §XVIII · pp. 43–44, Figure 34 · p. 44
  • Extended Reality integrates virtual entities with physical surroundings, while user interaction research targets subtle mobile interfaces, haptics, and brain-computer interfaces for fuller sensory experiences.BCIs could directly stimulate the cerebral cortex to simulate sensory experiences beyond current VR/AR limitations, while XR connects AI, computer vision, IoT, and other technologies.Source: §XVIII · p. 45
  • The technical agenda prioritises user-in-the-loop IoT and robotics, lightweight AI, faster and privacy-preserving blockchain, accurate computer vision, lower edge latency, secure edge computing, and cross-layer network adaptation.Targets include spatial, scene, body, and pose understanding; restoration of real-virtual content; 1 ms-class last-mile transmission; and network adaptation using application-level QoE feedback.Source: §XVIII · p. 45
  • Avatar research must support real-time expression, diverse device integration, nonhuman representations, and ethical digital selves whose persistent identities raise questions about rights, obligations, and inheritance.Content creation should be participatory and culturally diverse, with creator incentives, decentralised censorship governance, and preservation of digital heritage; virtual and real economies require integrated analysis and accurate incentive-based simulation.Source: §XVIII · p. 46
  • Social acceptability depends on privacy, diversity, fairness, addiction, cross-world norms, scalable autonomous evaluation, and strategies against cybercrime and abuse.Because manual matching of rules across the metaverse is infeasible, autonomous agents are needed to adopt and evaluate norms at scale.Source: §XVIII · p. 47
  • Metaverse security requires convenient alternative authentication and digital-asset protection, while privacy requires automated preference recognition and safeguards against persistent activity traces and manipulated digital copies.Biometric modalities such as muscle movements, gestures, and eye gazes still require improvements in security, accuracy, speed, and device acceptability.Source: §XVIII · p. 47
  • Trust and accountability require an innovation-responsive definition of personal data, including biometrically inferred data, and protection for minorities and vulnerable communities from the outset.Existing privacy regulations may not keep pace with XR data innovation, while mistreatment in immersive systems could be experienced as real-world harm.Source: §XVIII · p. 47

XIX. CONCLUDING NOTES

The metaverse requires a holistic development approach spanning technologies and ecosystems, while significant challenges remain before it becomes integrated with physical reality and everyday life.

Future outlook

  • Emerging technologies, powerful computing devices, and intelligent wearables are expected to make future virtual worlds more interactive, alive, embodied, and multimedia.Technology companies including Apple and Google are pursuing plans to materialise the metaverse.Source: §XIX · p. 47
  • Many challenges must still be overcome before the metaverse becomes integrated into the physical world and everyday life.Source: §XIX · p. 47

Research agenda

  • The survey calls for a holistic approach to building the metaverse as an enormous entity existing in parallel with physical reality.It synthesizes recent work across technologies and ecosystems to identify fundamental challenges and a research agenda for the coming decades.Source: §XIX · p. 47

Related papers

Loading 2110.05352v3…