PaperlayerView exact revision on arXiv

Source-linked AI summary

AI and 6G into the Metaverse: Fundamentals, Challenges and Future Research Trends

Muhammad Zawish, Fayaz Ali Dharejo, Sunder Ali Khowaja, Kapal Dev, Steven Davy, Nawab Muhammad Faseeh Qureshi, Paolo Bellavista

arXiv:2208.10921v2cs.AIcs.HCcs.NIcs.SI

TL;DR

The paper addresses the need for an integrated understanding of AI and 6G in building immersive Metaverse experiences. It surveys the relevant technologies, their joint roles, sustainability, applications, and research challenges, concluding that the area spans immersive services, ubiquitous intelligence, tactile feedback, and self-optimizing capabilities.

  • Problem

    Existing surveys address limited perspectives of AI, 6G, or Metaverse technologies, motivating a comprehensive review of their integrated role in immersive experiences.

  • Method

    The survey reviews AI, computer vision, learning paradigms, Edge AI, wireless technologies, AI–6G integration, sustainability, applications, projects, and future directions.

  • Results

    The review identifies AI and 6G capabilities supporting ubiquitous intelligence, tactile feedback, and self-optimizing Metaverse services, including holographic telepresence and remote surgery.

  • Takeaways & Limitations

    Metaverse realization depends on coordinating AI and 6G across immersive services while addressing sustainability and open research challenges.

Abstract

from arXiv · show

Since Facebook was renamed Meta, a lot of attention, debate, and exploration have intensified about what the Metaverse is, how it works, and the possible ways to exploit it. It is anticipated that Metaverse will be a continuum of rapidly emerging technologies, usecases, capabilities, and experiences that will make it up for the next evolution of the Internet. Several researchers have already surveyed the literature on artificial intelligence (AI) and wireless communications in realizing the Metaverse. However, due to the rapid emergence and continuous evolution of technologies, there is a need for a comprehensive and in-depth survey of the role of AI, 6G, and the nexus of both in realizing the immersive experiences of Metaverse. Therefore, in this survey, we first introduce the background and ongoing progress in augmented reality (AR), virtual reality (VR), mixed reality (MR) and spatial computing, followed by the technical aspects of AI and 6G. Then, we survey the role of AI in the Metaverse by reviewing the state-of-the-art in deep learning, computer vision, and Edge AI to extract the requirements of 6G in Metaverse. Next, we investigate the promising services of B5G/6G towards Metaverse, followed by identifying the role of AI in 6G networks and 6G networks for AI in support of Metaverse applications, and the need for sustainability in Metaverse. Finally, we enlist the existing and potential applications, usecases, and projects to highlight the importance of progress in the Metaverse. Moreover, in order to provide potential research directions to researchers, we underline the challenges, research gaps, and lessons learned identified from the literature review of the aforementioned technologies.

I. INTRODUCTION

The survey frames the Metaverse as an immersive convergence of XR, AI, cloud-edge computing, and B5G/6G, while identifying scalability, latency, and data-rate demands as central challenges. It addresses these gaps through a broad review of AI, wireless technologies, their integration, sustainability, applications, and future research.

  • Metaverse integrates XR, cloud and edge computing, social platforms, gaming, cryptocurrencies, and AI as an anticipated evolution of the Internet.
  • Existing infrastructure struggles with Metaverse scalability, while 5G cannot meet 0.1ms haptic-delay requirements and projected data rates above Tbps.
  • AI and 6G are needed to support Metaverse applications including virtual education, navigation, immersive gaming, remote healthcare, and realistic 3D or holographic content.
  • The survey reviews AI techniques, computer vision, learning paradigms, and Edge AI to derive communication requirements for Metaverse services.
  • It examines B5G/6G services for immersive experiences and holographic telepresence, including 5G-NR, URLLC, mmWave, MEC, and THz communications.
  • The survey further studies the integrated AI–6G role, sustainability, applications, ongoing projects, challenges, research directions, and lessons learned.

II. AI AND 6G FOR METAVERSE: BACKGROUND AND TECHNICAL ASPECTS

The background section describes the Metaverse as a computer-generated universe combining physical and digital interaction through technologies such as VR, AR, MR, spatial computing, AI, and blockchain. It also introduces AI advances and their relevance to visual content, intelligent systems, and Metaverse infrastructure.

  • The Metaverse is presented as a computer-generated universe beyond the physical world, incorporating 3D animation, VR, AR, spatial computing, blockchain, and related technologies.
  • NFTs are described as mechanisms for digital ownership in an open, decentralized environment, distinguishing the Metaverse’s economic vision from Web3’s service model.
  • Transformer scale has expanded from GPT’s 110 million parameters to a Google Brain transformer exceeding 1 trillion parameters, indicating continued growth of deep neural networks.
  • Metaverse environments combine shared virtual spaces, AR glasses, headsets, VR devices, smartphones, and wristbands for play, learning, creation, commerce, and communication.
  • The paper positions AI, deep learning, and machine learning as foundations for Metaverse infrastructure and potentially advanced man–machine intelligence.
  • The background highlights deep learning for computer-vision tasks such as 2D and 3D human-pose tracking in VR, AR, and XR applications.

B. Spatial Computing

Spatial computing integrates AR, VR, and MR to digitalize interactions among people, machines, objects, and their locations. The section presents these technologies as foundations for transforming physical environments into interactive digital spaces, while highlighting synchronization and mobility challenges.

  • B. Spatial Computing: Spatial computing integrates AR, VR, and MR in a three-dimensional world to improve interactions and industrial activities.It supports applications including training and modeling and enables interaction with digital objects.
  • 1) Virtual Reality (VR): VR immerses users in fully synthetic, computer-generated environments explored through 360° interaction.Head and body tracking links users to virtual environments distinct from physical surroundings.
  • B. Spatial Computing: Metaverse environments require users to synchronize dynamic events, while cellular networks support smooth outdoor mobility.The cited discussion contrasts evolving 5G and 6G capabilities with earlier network generations.
  • 2) Augmented Reality (AR): AR enhances physical surroundings by overlaying digital visual, audio, and other sensory elements.Applications include gaming, directions, and locating objects.
  • 3) Mixed Reality (MR): MR combines physical and digital items that coexist and interact in real time, functioning as a hybrid of AR and VR.The section describes MR as linking physical space, user interaction, and virtual entities.

2) 6G and Internet of Everything:

The section surveys how 6G, AI, edge infrastructure, blockchain, and related technologies could support scalable Metaverse services. It emphasizes high-speed, low-latency connectivity, intelligent infrastructure, accessibility, personalization, and democratization.

  • 2) 6G and Internet of Everything: 6G is envisioned as an Internet of Everything platform integrating users, devices, vehicles, and environments with substantially higher speed and capacity.The discussion describes projected terahertz communication and theoretical speeds thousands of times faster than existing optical fibers.
  • 2) 6G and Internet of Everything: 6G-enabled terahertz communication and virtualized services could support real-time 3D imagery, holographic communication, and autonomous-driving applications.The cited passage associates 6G with extremely low latency and 3D XR experiences.
  • 2) 6G and Internet of Everything: AR/VR/MR adoption remains limited, while 5G and edge computing can address some of its deployment challenges across industry verticals.The survey summarizes implementations and use cases in its state-of-the-art table.
  • III. ROLE OF AI IN METAVERSE: Metaverse infrastructure must accommodate massive machine-type communication, AI model execution, temporary storage, and seamless service operation.The AI-based infrastructure layer supports decision-level operations across the Metaverse architecture.
  • III. ROLE OF AI IN METAVERSE: Blockchain-based decentralization does not by itself guarantee democratization, which requires equal opportunity for users and creators.The discussion uses a constrained NFT purchase example to distinguish ownership mechanisms from broader participation.
  • III. ROLE OF AI IN METAVERSE: AI can improve accessibility and personalization through computer vision, automated voice, social avatars, brain-computer interfaces, and real-time user analytics.Personalized experiences may account for emotions, mental well-being, disabilities, and application context.

A. Learning Paradigms for Metaverse

The section contrasts supervised learning with emerging diversified and self-supervised approaches for Metaverse applications. It also describes large-scale VR data use in training digital agents and unified multimodal models.

  • A. Learning Paradigms for Metaverse: Supervised learning has advanced Metaverse applications but tends to make them task dependent because it relies heavily on human supervision.The passage identifies reinforcement learning and other diversified strategies as alternatives under exploration.
  • A. Learning Paradigms for Metaverse: Meta AI is exploring self-supervised unified multimodal models, while Google DeepMind used VRChat data to train a digital agent to interact with humans.The passage reports 16 million hours of VR data generated in 2018 and 20,000 hours used for training.

B. Computer vision for Metaverse

Computer vision supplies realistic avatars, faces, and holograms for Metaverse experiences, while AI and 6G must address the scalability, latency, reliability, and bandwidth demands of these services.

  • B. Computer vision for Metaverse: AI computer vision supports animated 3D human models, realistic faces, and hologram creation for Metaverse applications.Examples include volumetric capture and model-generation systems such as Kinetix, Mixamo, and StyleGAN.
  • B. Computer vision for Metaverse: Metaverse services require scalable infrastructure, low latency, stable connectivity, security, and higher throughput than current LTE and 5G systems can consistently provide.The survey identifies these requirements for applications ranging from virtual education and gaming to remote healthcare.
  • B. Computer vision for Metaverse: 5G can satisfy some applications requiring 100 Mbps to a few Gbps, at least 99% reliability, and delays below 5 ms, but broader Metaverse scenarios demand more.Trials reported device rates of 1–4.5 Gbps with delay below 5 ms.
  • B. Computer vision for Metaverse: 6G is positioned to support AI-based Metaverse services through 1 Tbps-class rates, 0.1 ms end-to-end delay, and extremely reliable low-latency communication.These capabilities target holographic, haptic, XR, and other data-intensive applications.

C. Immersive Experiences over Wireless

Wireless immersive experiences depend on adaptive XR streaming, high-bandwidth links, and edge computation to deliver omnidirectional content under tight latency constraints.

  • C. Immersive Experiences over Wireless: Immersive wireless streaming may require omnidirectional visual content with latency below 20 ms and a smooth, reliable transmission mechanism.Dynamic streaming frameworks adapt XR bitrate to network bandwidth and user geolocation.
  • C. Immersive Experiences over Wireless: mmWave communications offer high transmission rates and low delay for wireless VR, while MEC addresses the computational burden of viewport rendering.VR rendering remaps pixels from a viewing sphere to a 2D viewport using intensive matrix operations.
  • C. Immersive Experiences over Wireless: Field-of-view mechanisms can reduce 360° video bandwidth requirements by up to 80% compared with transmitting the full video.FOV prediction, frame segregation, and multicasting are proposed to exploit user viewpoints.
  • C. Immersive Experiences over Wireless: Combining mmWave, sub-6GHz, MEC, and FOV-aware streaming can lower bandwidth requirements and transmission delay versus conventional 360° video delivery.One scheme encodes the viewport and renders standby tiles at different quality levels; another sends base and enhancement layers over different links.
  • C. Immersive Experiences over Wireless: THz communication is explored for immersive systems because existing mmWave technology cannot meet the rapidly increasing bandwidth demand of Metaverse applications.A DRL-based THz/RIS strategy uses current and historical user viewpoints to improve indoor VR downlink transmission and long-term QoE.

D. Holographic Telepresence in Metaverse

Holographic telepresence and related Metaverse services require ubiquitous, human-centric 6G connectivity that integrates physical and cyber spaces with extreme latency, reliability, and coverage targets.

  • D. Holographic Telepresence in Metaverse: Holographic communication would let users view high-quality three-dimensional digital representations without wearing head-mounted displays.B5G/6G is expected to support this through ultra-low latency, ultra-high data rates, and high reliability.
  • D. Holographic Telepresence in Metaverse: Current 5G trials achieved average device data rates of 1–4.5 Gbps with latency below 5 ms, supporting many existing AR/VR requirements.The cited analysis associates these results with applications needing 100 Mbps to a few Gbps, below-5-ms latency, and at least 99% reliability.
  • D. Holographic Telepresence in Metaverse: 6G is expected to extend Metaverse access beyond 5G’s coverage gaps in remote areas, oceans, mountains, forests, and airspace.The proposed global coverage supports real-time data collection, transport, and utilization.
  • D. Holographic Telepresence in Metaverse: A wireless isochronous real-time 6G framework reported round-trip latency below 0.1 ms and outage probability below 10^-6.Its cycle times were reported as 10x shorter than the 5G latency target, supporting novel avatar communication modalities.

B. AI-led Autonomous 6G Networks for Metaverse

AI is presented as necessary for autonomous 6G operation in the Metaverse, while sustainability analysis considers virtual substitutes, environmental effects, and social inclusion.

  • B. AI-led Autonomous 6G Networks for Metaverse: 6G networks require intelligent autonomous adjustment to meet changing coverage, latency, reliability, security, and user-experience requirements at large scale.Existing rule-based networks cannot self-evolve effectively and depend heavily on manual intervention.
  • B. AI-led Autonomous 6G Networks for Metaverse: Deep reinforcement learning and federated learning are identified as AI approaches for self-healing, self-optimizing, and self-organizing Metaverse networks.Federated architectures can support distributed training where users do not access one another’s data.
  • B. AI-led Autonomous 6G Networks for Metaverse: Haptic Metaverse services require roughly 1 ms delay, motivating B5G/6G support beyond the approximately 25 ms latency of pre-5G networks.The tactile Internet targets real-time human-to-human and human-to-machine interaction through rapid action-feedback cycles.
  • B. AI-led Autonomous 6G Networks for Metaverse: Metaverse sustainability is discussed across virtual resources, virtual travel, digital twins, psychological barriers, and social sustainability.The survey emphasizes both environmental and social dimensions, including equity, inclusiveness, and accessibility.
  • B. AI-led Autonomous 6G Networks for Metaverse: Replacing physical products with digital alternatives is presented as a potential route to lower resource use and emissions.One cited example reports that virtual denim could reduce CO2 emissions by 10%, alongside substantial water savings.

B. Travelling through Metaverse

The Metaverse may replace some business and recreational travel through immersive virtual meetings and concerts, while digital twins support environmental sustainability and climate awareness.

  • B. Travelling through Metaverse: Virtual meetings and concerts indicate that the Metaverse could replace business and recreational travel to a certain degree.Immersive experiences may also support work-from-home practices that reduce CO2 emissions.
  • B. Travelling through Metaverse: Digital twins transform real-world entities into virtual representations for climate study, policy-making, and sustainability across supply chains, manufacturing, and healthcare.Applications include studying human activities and climate impacts, optimizing logistics and materials, and forecasting health in relation to air quality.
  • B. Travelling through Metaverse: Building-operation digital twins improved space utilization by 25%, human productivity by 20%, maintenance and operational efficiency by 35%, and reduced building carbon emissions by around 50%.
  • B. Travelling through Metaverse: Metaverse simulations of climate change may encourage pro-environmental actions and reduce carbon footprints by increasing public awareness.The cited work connects immersive climate experiences with choosing more environmentally friendly products.
  • B. Travelling through Metaverse: Sustainability in the Metaverse also includes social rights such as diversity, accessibility, and equity, requiring collaborative efforts to reduce bias and democratize Web3.

A. What Are the Use Cases of the Metaverse

The Metaverse offers use cases spanning digital interaction, marketing, blockchain, tourism, communication, remote work, healthcare, education, and gaming.

  • A. What Are the Use Cases of the Metaverse: Metaverse users can navigate immersive 3D spaces as avatars while creating, sharing, and trading virtual experiences and assets.
  • A. What Are the Use Cases of the Metaverse: Use cases include new marketing opportunities, blockchain-based dApps and NFTs, virtual tourism, and peer-to-peer real-time communication.These applications support virtual-world engagement, digital transactions, immersive destinations, and direct browser communication.
  • A. What Are the Use Cases of the Metaverse: Metaverse environments can support virtual offices and learning spaces where remote participants work or learn together as if in the same room.
  • A. What Are the Use Cases of the Metaverse: Healthcare applications enable geographically independent patient-professional interaction and immersive virtual-reality training for medical students.
  • A. What Are the Use Cases of the Metaverse: Gaming applications combine play-to-earn models with tradable in-game assets, linking decentralized blockchain benefits to economic rewards.

3) Completely New Economy:

The Metaverse supports creator economies and virtual asset trading through blockchain-based platforms, games, marketplaces, and major technology-company projects.

  • 3) Completely New Economy:: Creator economies allow users to trade assets across Metaverse spaces, including NFTs created within Metaverse games, with growth linked to DeFi, NFTs, and blockchain games.
  • 3) Completely New Economy:: Ongoing projects include Facebook, Epic Games, Microsoft Mesh, and Decentraland, illustrating involvement from technology, gaming, enterprise, and blockchain platforms.
  • 3) Completely New Economy:: Decentraland enables virtual real-estate trading and NFT creation, with one reported auction involving virtual real estate worth more than 2 million.
  • 3) Completely New Economy:: Decentraland operates through a DAO and smart contracts, allowing users to vote on aspects of the Metaverse and purchase assets using currencies including Polygon and Ethereum.
  • 3) Completely New Economy:: Platforms such as Oculus and Enjin provide access to VR experiences or NFT creation, integration, and trading for Metaverse applications.

D. Silks

The surveyed projects illustrate Metaverse applications in play-to-earn gaming, education, healthcare, business, immersive commerce, and 3D avatar creation, alongside unresolved scalability and accessibility challenges.

  • D. Silks: Silks uses a blockchain-enabled play-to-earn economy in which users own, trade, and interact with digital assets representing thoroughbred racehorses.
  • D. Silks: The XR initiative explores interoperable XR and VR applications beyond gaming, including education and healthcare training.
  • D. Silks: MirageXR enables holographic training programs, performance analytics, and recorded practice sessions for workplace learning and industrial training.
  • D. Silks: Highstreet combines browser-accessible virtual commerce, face-to-face interaction, cryptocurrency exchange, and simultaneous digital and physical product presentation.
  • D. Silks: Metahero uses 3D scanning and NFT smart contracts to create realistic meta-avatars and meta-objects from physical bodies and objects.
  • D. Silks: Future Metaverse integration raises challenges in heterogeneity, bandwidth management, energy efficiency, security, and access to powerful AI models on handheld devices.

3) Democratization:

The Metaverse’s democratization depends on equitable access, trustworthy governance, and infrastructure that can support diverse users and uninterrupted immersive services. Space-air-ground-sea integration expands access but introduces bandwidth, delay, reliability, and evaluation challenges.

  • 3) Democratization:: Metaverse democratization remains incomplete because AI-based anomaly-detecting smart contracts are not yet realized and AI bias still requires human involvement.The passage frames democratization as an ongoing requirement for sustainability as user populations grow and diversify.
  • 3) Democratization:: Space-air-ground-sea networks can provide ubiquitous access and seamless movement across integrated networks, supporting uninterrupted immersive experiences.Non-terrestrial networks provide access from anywhere, while terrestrial networks support transitions among integrated segments.
  • 3) Democratization:: Frequent data exchange across integrated networks can cause significant delays, consume substantial bandwidth and backhaul resources, and compromise reliability because of satellite mobility.The paper calls for new architectures, simulation platforms, and reliability metrics for Metaverse services.
  • 3) Democratization:: AI and 6G are positioned to support immersive applications, but existing infrastructure must scale to massive device populations and stringent tactile-feedback requirements.Applications such as remote healthcare, teleoperations, and telepresence depend on reliable low-latency communication and processing.

2) AI-based Network Automation for Metaverse:

AI-based network automation is presented as a response to Metaverse requirements for adaptive, scalable, secure, and low-latency services. Its deployment remains constrained by changing data needs, device resources, hardware availability, energy use, and broader sustainability and privacy concerns.

  • 2) AI-based Network Automation for Metaverse:: Metaverse network automation must support heterogeneous users and services through self-allocation, self-configuration, and self-optimization of network resources.The paper identifies AI as central to enabling self-adaptive capabilities in 6G networks.
  • 2) AI-based Network Automation for Metaverse:: Changing contexts create a shortage of labeled training data for updating AI models.The passage identifies this as a direct challenge for AI-based network automation.
  • 2) AI-based Network Automation for Metaverse:: Deep-learning models can be too complex to deploy on resource-constrained devices.This limits local execution of intelligent Metaverse services.
  • 2) AI-based Network Automation for Metaverse:: Limited hardware availability constrains support for new and emerging intelligent Metaverse services.The paper lists hardware support as a separate infrastructure challenge.
  • 2) AI-based Network Automation for Metaverse:: Self-supervised learning and distributing models across the end-edge-cloud continuum are proposed to reduce training and computational burdens.The proposed direction combines alternative training with workload distribution across infrastructure layers.
  • 2) AI-based Network Automation for Metaverse:: Metaverse sustainability requires managing energy-intensive local intelligence, data centers, blockchain processes, privacy exposure, and unequal access.The paper also identifies governance, audit, and equitable distribution as sustainability concerns.

X. LESSONS LEARNED

The paper’s lessons learned connect AI-native 6G capabilities with immersive Metaverse services while emphasizing sustainability, security, and scalability. It presents virtual interaction and intelligent networking as opportunities, but stresses that governance, privacy, energy use, and equitable access remain necessary conditions.

  • 2) Circular Economy:: Metaverse applications can reduce travel through virtual conferences and exhibitions, supporting circular-economy goals and cost savings.The passage also describes games that could motivate reduced plastic use and recycled-product consumption.
  • 3) Sustainability in Metaverse:: Sustainability benefits are conditional because Metaverse infrastructure, NFTs, and blockchain can increase energy consumption without suitable regulation and design choices.The paper mentions hyperscale data centers, proof-of-stake transactions, and digital twins as relevant sustainability considerations.
  • 5) Wireless Interactivity in Metaverse:: Future 6G systems are expected to combine connectivity with physical and virtual spaces, while AI becomes native and ubiquitous across the network.THz communication is identified as useful for ultra-high-rate short-distance communication and immersive interactivity.
  • XI. CONCLUSION: Large-scale Metaverse deployment requires infrastructure beyond current limited scaling capabilities, including AI-automated wireless networks and reinforcement learning.The paper notes that Facebook Space could handle only three people in its cited environment.
  • XI. CONCLUSION: The survey concludes that AI and 6G jointly support ubiquitous intelligence, tactile feedback, and self-optimizing capabilities across Metaverse services.The reviewed applications range from holographic telepresence to remote surgery.
Loading 2208.10921v2…