PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Immersive Communications in 6G

Xuemin Shen, Jie Gao, Mushu Li, Conghao Zhou, Shisheng Hu, Mingcheng He, Weihua Zhuang

arXiv:2303.05283v1cs.NIeess.SP

TL;DR

Immersive communications pose major rate, delay, reliability, synchronization, and computing challenges for 6G networks. This survey synthesizes use cases, concepts, requirements, solutions, and future directions across XR, haptic, and holographic communication. It concludes that substantial resources and new paradigms are needed, while identifying limits to the applicability of current time-sensitive networking efforts.

  • Problem

    Immersive communications place demanding requirements on transmission rate, latency, reliability, synchronization, and computing that 5G cannot fully support.

    Source: §I · p. 2, §VI · p. 21

  • Method

    The survey reviews use cases, concepts, procedures, network requirements, challenges, solutions, and future directions for XR, haptic, and holographic communication.

    Source: §I · p. 2, §VI · p. 21

  • Results

    The survey concludes that immersive communications’ rate, delay, reliability, multimodal, and computing demands call for substantial communication and computing resources and paradigms such as AI-native communication, multi-tier computing, and user-centric networking.

    Source: §VI · p. 21

  • Takeaways & Limitations

    The survey identifies advances across network layers and paradigms such as QoE-oriented networking and AI-native communications as promising directions for further research.

    Source: §VI · p. 21

    Current time-sensitive and deterministic networking efforts largely target link and network layers and industrial networks, so their solutions may not readily apply to every immersive-communication use case.

    Source: C. Time-Sensitive and Deterministic… · p. 20

Abstract

from arXiv · show

The sixth generation (6G) networks are expected to enable immersive communications and bridge the physical and the virtual worlds. Integrating extended reality, holography, and haptics, immersive communications will revolutionize how people work, entertain, and communicate by enabling lifelike interactions. However, the unprecedented demand for data transmission rate and the stringent requirements on latency and reliability create challenges for 6G networks to support immersive communications. In this survey article, we present the prospect of immersive communications and investigate emerging solutions to the corresponding challenges for 6G. First, we introduce use cases of immersive communications, in the fields of entertainment, education, and healthcare. Second, we present the concepts of immersive communications, including extended reality, haptic communication, and holographic communication, their basic implementation procedures, and their requirements on networks in terms of transmission rate, latency, and reliability. Third, we summarize the potential solutions to addressing the challenges from the aspects of communication, computing, and networking. Finally, we discuss future research directions and conclude this study.

I. INTRODUCTION

Immersive communications promise more lifelike interaction through 3D imagery, haptics, and links between physical and virtual worlds, but their demands challenge 5G. This survey reviews use cases and the concepts, requirements, challenges, solutions, and future directions for 6G.

I. INTRODUCTION

  • Immersive communications add 3D images with parallax, haptic information, and interactions across physical and virtual worlds, supporting use cases from telepresence to industrial collaboration.The authors describe potential effects on how people study, work, and entertain.Source: §I · p. 1
  • Immersive communications can demand terabits-per-second transmission for live 3D imagery, millisecond haptic delays, synchronized data streams, and substantial storage and processing.The article argues that these requirements exceed 5G capabilities and may require integrated eMBB and URLLC support.Source: §I · p. 2
  • The survey reviews XR, haptic, and holographic communications across use cases, implementation procedures, network requirements, technical challenges, potential solutions, and open research issues.It takes communication, networking, and computing perspectives and focuses on recent work.Source: §I · p. 2

II. USE CASES

  • The article introduces representative immersive-communication use cases across commercial and enterprise scenarios, ranging from gaming to industrial control.It identifies four representative use cases and refers readers to Table II for a list.Source: §II · p. 2

A. Immersive Gaming and Entertainment

The survey illustrates immersive communications through use cases spanning entertainment, healthcare, education, holographic collaboration, and persistent virtual spaces. These examples combine XR, haptics, holography, and real-time or asynchronous interaction.

A. Immersive Gaming and Entertainment

  • XR gaming lets players interact with virtual environments and one another; haptic devices can capture player actions, provide mapped sensations, and control or synchronize objects.Wireless devices are presented as a way for players to move freely during XR experiences.Source: A. Immersive Gaming and Entertainment · pp. 2–3

B. Telesurgery

  • Telesurgery uses remote robotic arms, low-latency surgical video, and haptic feedback to convey tissue texture and suture tension between surgeons and operating sites.AR can overlay medical images on surgical video, while robotic arms can cancel physiological hand tremors.Source: B. Telesurgery · p. 3

C. Immersive Learning

  • Immersive learning combines XR and haptics for interactive instruction, with real-time feedback for skills practice or record-and-replay for asynchronous learning.Teachers can observe learners’ actions and offer corrections remotely; recorded demonstrations let more students learn at their own pace.Source: C. Immersive Learning · p. 3

D. Holographic Teleconference

  • Holographic teleconferencing reconstructs remote participants as 3D audiovisual holograms and can add haptic signals such as a sensed handshake.Multiple sensors capture a participant’s 3D visual and audio information for transmission and reconstruction.Source: D. Holographic Teleconference · pp. 3–4

E. Metaverse

  • A metaverse synchronizes activity from the physical world into digital spaces and interactions from digital spaces back to the physical world.Sensors and actuators capture activity for avatars, while users interact with avatars, objects, and services through XR devices.Source: E. Metaverse · p. 4

III. IMMERSIVE COMMUNICATIONS: CONCEPTS AND REQUIREMENTS

The paper frames immersive communications as lifelike interaction through 3D audiovisual and/or haptic exchange in physical or virtual settings. It reviews XR, haptics, and holography, with XR spanning different levels of virtuality.

III. IMMERSIVE COMMUNICATIONS: CONCEPTS AND REQUIREMENTS

  • Because immersive communications lack a commonly agreed definition, the paper treats them as a paradigm and supporting technologies for lifelike interaction through 3D audiovisual and/or haptic exchange.The interaction may take place in the physical world, the virtual world, or both.Source: §III · p. 4
  • The paper focuses on XR, haptic communication, and holographic communication, which may coexist in a use case and are not necessarily an exhaustive set of forms.It aims to sketch their concepts, basic implementation procedures, and network requirements; representative use-case requirements are illustrated in Fig. 2 and summarized in Table IV.Source: §III · p. 4

A. Extended Reality

  • XR combines physical and virtual worlds through video processing and data fusion, with AR at the lower-virtuality end, VR at the higher end, and MR combining aspects of both.XR devices use sensory data and virtual environments to create artificial scenes on headsets or portable displays.Source: 1) Concept · p. 4, A. Extended Reality · p. 4

1) Concept:

  • VR has advanced alongside computing and display technologies, while high-virtuality content still requires substantial computation and is commonly processed and rendered on user devices.Wireless VR is a research focus because a wired headset-to-console connection restricts users; multi-sensory XR integrates visual, auditory, olfactory, and tactile perception.Source: 1) Concept · pp. 4–5

2) Basic Implementation Procedure:

XR implementation involves content transmission, rendering, and feedback collection, with networks supporting each step. VR commonly delivers field-of-view tiles, while AR processing can combine device capture with network-side computation.

2) Basic Implementation Procedure:

  • XR procedures can be organized as content transmission, rendering, and feedback collection, with communication networks playing a role in each step.The steps apply across XR technologies with different levels of virtuality.Source: 2) Basic Implementation Procedure · p. 5
  • VR tile-based delivery divides equirectangular video into spatiotemporal chunks and sends tiles within the user’s field of view, reducing transmitted data size.The server selects tiles based on the user’s viewpoint and network conditions; AR devices can upload captured frames for server processing and receive processed content.Source: 2) Basic Implementation Procedure · pp. 5–6
  • VR rendering stitches received tiles and projects stereoscopic video for both eyes, with workload potentially offloaded to nearby edge servers.AR processing includes location tracking, mapping, object recognition, augmentation, and rendering; some or all of this work can be offloaded to network servers.Source: 2) Basic Implementation Procedure · p. 6
  • XR devices collect actions and motions through sensors or cameras, while VR also gathers viewpoints to guide which tiles are delivered.Motion emulation can support proactive VR content requests, and interactive applications can process user inputs locally or upload them for server-side updates.Source: 2) Basic Implementation Procedure · pp. 6–7

3) Requirements:

XR faces stringent motion-related latency and high delivery-rate demands, while haptic communication adds touch-based interaction whose implementation varies with the number of participants.

3) Requirements:

  • VR motion-to-photon delays above 20 ms may cause sickness, while current industries target below 15 ms; AR interaction rendering requires 75 ms for gaming and 250 ms for telemetry.XR also needs high delivery rates, with VR video requirements of 2.35 Gbps or above, and network computing capacity can bottleneck interactive applications.Source: 3) Requirements · p. 7

1) Concept:

  • Haptic communication exchanges touch information over a network, including tactile sensations such as texture and temperature and kinesthetic information about limb position, motion, and force.Haptic interfaces capture and display this information through sensors and actuators; graspable, wearable, and touchable interfaces serve different combinations of tactile and kinesthetic information.Source: 1) Concept · pp. 7–8

2) Basic Implementation Procedure:

  • Bilateral haptic communication acquires tactile and kinesthetic data, reduces it, transmits it between interfaces, and displays it through haptic actuators.Reduction may use compression or predictive coding; transmitted substreams require suitable quality of service and synchronization, and receiver-side prediction can help when data is delayed or unreliable.Source: 2) Basic Implementation Procedure · p. 8, Figure 5 · p. 9
  • Multilateral haptic communication adds information sharing, fusion of users’ haptic inputs, and distribution tailored to affected users.These additions support cooperative tele-operation, where users may share information without direct interaction and may need different reaction forces.Source: 2) Basic Implementation Procedure · pp. 8–9, Figure 6 · p. 9

3) Requirements:

Haptic network requirements depend on the volume and timing of haptic data, the interaction scenario, and whether data reduction is used.

3) Requirements:

  • Haptic data rates depend on packet rate and packet size: updates typically exceed 1,000 per second, while packet sizes are about 8, 80, and 800 bytes for 1-, 10-, and 100-DoF data.A hand’s kinesthetic data can use a 24-DoF model, and tactile data can involve even higher DoF.Source: 3) Requirements · p. 9
  • Delay needs vary with perception and interaction dynamics: high-, medium-, and low-dynamics tasks require 1–10 ms, 10–100 ms, and 100–1,000 ms, respectively.Remote machine manipulation requires below 2 ms, compared with below 50 ms for monitoring.Source: 3) Requirements · pp. 9–10
  • Required reliability ranges from above 99.9% for immersive gaming to above 99.999% for telesurgery and remote machine manipulation, with data reduction also requiring above 99.999% in gaming.The reliability measure may be bit error rate, packet loss rate, delay-bound violation probability, or prediction error.Source: 3) Requirements · p. 10

C. Holography and Holographic Communication

Holographic communication transfers and renders 3D scene data, with this review focusing on autostereoscopic displays; its general pipeline captures, processes, transmits, and renders scene data.

1) Concept:

  • Holography ranges from optical and digital methods to computer-generated holography, while alternative “false holography” approaches create 3D display illusions using methods such as volumetric display.Computer-generated holography digitally creates both the interference pattern and displayed 3D image, but has high computational complexity; generating dynamic 3D holograms in real time remains challenging.Source: 1) Concept · p. 10
  • In this review, holographic communication transfers data representing dynamic 3D images for autostereoscopic display, viewable without eyewear or headsets and ideally varying with the viewer’s position or angle.The receiver may use real holography, volumetric display, or another technique that achieves autostereoscopic 3D display.Source: 1) Concept · p. 10

2) Basic Implementation Procedure:

  • The general pipeline captures a scene, processes sensor output into a composite 3D representation, compresses and transmits it, then decodes, reconstructs, and renders it at the receiver.Capture may use camera arrays or LIDAR; processing may compute depth and fuse images, and can involve an edge server.Source: 2) Basic Implementation Procedure · p. 11, Figure 7 · p. 11
  • Displays approximate an ideal recreated light field using devices such as cylindrical light-field, persistence-of-vision, or static volumetric displays; audio capture and synchronization may also support immersion.These display approaches create 3D-image illusions through different mechanisms, while audio-video synchronization matters when sound is included.Source: 2) Basic Implementation Procedure · p. 11

3) Requirements:

Holographic content can demand very high rates and synchronization, while XR solutions target timely delivery by selecting content and adapting network and computing resources.

3) Requirements:

  • Human-sized volumetric holograms require 300 Mbps to 3 Gbps at 30 frames per second, while light-field holograms require 100 Gbps to 2 Tbps.Real-time holographic communication should keep overall capture, processing, transmission, and rendering delay below 100 ms; multisource and multisensory synchronization also matter.Source: 3) Requirements · pp. 11–12

A. Extended Reality

  • XR’s central challenge is delivering content on time under limited network transmission resources and computing capacity, motivating solutions in content selection, transmission, and computing optimization.Content selection minimizes delivered data size at the cost of tolerable performance degradation.Source: A. Extended Reality · p. 12, 1) Content Selection · p. 12, §IV · p. 12

1) Content Selection:

  • VR systems predict user viewpoints to choose tiles for proactive delivery, while AR systems adapt the frame rate of content offloaded for server-side processing.VR prediction may use local or server-based learning, but imperfect prediction and changing network conditions require adaptive resource management; AR offloading trades bandwidth against detection accuracy and playback fluency.Source: 1) Content Selection · pp. 12–13
  • XR digital twins can synchronize physical-world models with XR content, and proposed approaches adjust synchronization rates or allocate sliced network resources for fidelity and user experience.Synchronization across the physical world, digital twins, and XR content requires considerable network resources.Source: 1) Content Selection · p. 13

2) Transmission Improvement:

VR delivery is bottlenecked by limited data rates, while XR performance also depends on computing capacity. The section surveys transmission techniques and computing strategies for meeting these demands.

2) Transmission Improvement:

  • mmWave and THz can raise VR data rates, but their line-of-sight links are vulnerable to blockage; backup sub-6 GHz bands and multiple edge-server links address outages.Band switching can make rates time-varying, so content processing policies may be adjusted to compensate for transmission delays.Source: 2) Transmission Improvement · p. 13
  • Network virtualization can tailor VR routing to delivery mode, while QUIC can prioritize important tiles to reduce missing tiles during playback.WiFi-7 also targets high-throughput, low-latency XR through band aggregation, MIMO enhancements, and multi-AP coordination.Source: 2) Transmission Improvement · pp. 13–14

3) Computing Optimization:

  • MEC can project monoscopic video into stereoscopic video, cache processed XR content, and use deep reinforcement learning to allocate edge computing resources for requests.The surveyed work reports lower bandwidth consumption than cloud delivery of stereoscopic video and lower computing time than local projection.Source: 3) Computing Optimization · p. 14
  • Multi-tier computing coordinates cloud, fog, and edge resources to distribute processing and use storage, but its many offloading and caching choices complicate optimization.The surveyed work explores optimization using static network information and, for dynamic settings, digital twins with meta-learning.Source: 3) Computing Optimization · pp. 14–15
  • XR computing can also be optimized through nearby-user content aggregation, adaptive collaborative rendering, and device-to-device sharing of lightweight preprocessing tasks.Collaborative rendering arranges edge and device subtasks according to network characteristics to support parallel computing.Source: 3) Computing Optimization · p. 15

B. Haptic Communication

Haptic communication must deliver data with stringent delay and reliability, particularly at high packet rates. The section reviews data reduction methods and the conditions that shape their use.

B. Haptic Communication

  • Haptic communication faces stringent delay and reliability requirements, addressed through data reduction, communication and networking techniques, and prediction to compensate for delay or packet loss.Data reduction targets packet size or rate.Source: B. Haptic Communication · p. 15

1) Haptic Data Reduction:

  • Dense e-skin deployment can reach 25 sensors or actuators per cm^2, while haptic packet rates can exceed 1,000 packets per second.Together, many devices and high packet rates can drive high transmission-rate requirements.Source: 1) Haptic Data Reduction · p. 15
  • Haptic data size can be reduced through differential floating-point transmission or frequency-domain quantization, while perceptual masking supports sending updates only when differences exceed a threshold.The threshold can be the just-noticeable difference; time- and frequency-domain masking can also be combined.Source: 1) Haptic Data Reduction · p. 15
  • Reduction methods must fit the data type and service requirements: frequency-domain reduction adds processing delay, while reduction can make 99.999% reliability for telesurgery difficult to meet.Real-time time-domain reduction suits low-delay-tolerance uses such as immersive gaming.Source: 1) Haptic Data Reduction · pp. 15–16

2) Communication and Networking Solutions:

Haptic networking methods target delay and reliability, while prediction can compensate for delay or packet loss. The section surveys communication, resource-reservation, and prediction approaches.

2) Communication and Networking Solutions:

  • Haptic delay can be reduced with data preemption, NOMA, and grant-free scheduling, while interference management can limit retransmission delays.Grant-based scheduling can take 0.3–0.4 ms for scheduling-request and transmission-grant exchange.Source: 2) Communication and Networking… · p. 16
  • Reliability approaches include short block-length error-correcting codes, spatial diversity, and packet repetition; network slicing can reserve resources for haptic traffic.Slice reservations may account for tolerable delay, operator stability, and learned device traffic patterns.Source: 2) Communication and Networking… · p. 16

3) Haptic Data Prediction:

  • A 10 ms delay requirement limits HI separation to under 3,000 km, and packet loss or delay violations cannot be eliminated, motivating haptic data prediction.Prediction models can use historical haptic or correlated data, including force feedback, commands, and audiovisual inputs.Source: 3) Haptic Data Prediction · p. 16
  • Prediction can run at the receiver or transmitter to compensate for delay or packet loss, with network edge servers or nearby robots supporting computing-intensive prediction.Transmitters can also send updates only when prediction deviation exceeds the receiver’s just-noticeable difference.Source: 3) Haptic Data Prediction · pp. 16–17

C. Holographic Communication

Holographic communication can provide detailed 3D views from multiple viewpoints, but its data-rate and delay demands are substantial. The section surveys content processing and communication and networking approaches.

C. Holographic Communication

  • Holographic communication’s stringent data-rate and delay requirements arise alongside the large data volumes needed for detailed, continuous views from multiple viewpoints.Depending on transmitted data type, demand can range from hundreds of Mbps to several Tbps.Source: C. Holographic Communication · p. 17

1) Content Selection, Compression and Prediction:

  • Viewpoint-based selection can omit hologram portions unlikely to be observed, but immersive selection must account for six degrees of freedom and may require full-body or eye tracking.Tracking is challenging without head-mounted devices such as VR headsets.Source: 1) Content Selection, Compression and… · p. 17
  • Compression approaches exploit point-cloud inter-frame redundancy and correlations among views; MPEG and JPEG efforts address point-cloud and hologram representation formats.The surveyed methods include lossy real-time color encoding and multi-view coding for light-field video streaming.Source: 1) Content Selection, Compression and… · p. 17
  • Receiver-side prediction from object history can generate replacement packets after loss, reducing retransmission delay in holographic communication.Examples include LSTM-based prediction of human actions and movements in 3D.Source: 1) Content Selection, Compression and… · p. 17

2) Communication and Networking Solutions:

  • MEC can process captured data before rendering, avoiding the high communication delay of sending massive data to cloud servers; multi-tier computing can combine cloud, MEC, and local resources.Split rendering lets an MEC server and local device cooperatively decode and render holograms.Source: 2) Communication and Networking… · pp. 17–18
  • Transport approaches include QUIC-based streaming, flow-level control for substreams with different QoS needs, and selective retransmission of important hologram data.One example prioritizes retransmission of data for the center of the user’s field of view.Source: 2) Communication and Networking… · p. 18
  • THz communications could support Tbps-level holographic rates, but sustainable line-of-sight links require further investigation; visible light communication offers another high-rate option with indoor positioning.The surveyed work also studies coordination of THz and visible light communication for reliable high-data-rate service.Source: 2) Communication and Networking… · p. 18

V. IMMERSIVE COMMUNICATION: OPEN ISSUES AND FUTURE DIRECTIONS

The section surveys open issues for immersive communications, spanning synchronized multimodal delivery, AI-enabled functions, deterministic networking, and personalized QoE. It outlines research directions that adapt network and computing resources to these demands.

A. Multi-Modal Communications

  • Cross-modal and intra-stream synchronization both matter, while cross-layer coordination can use network conditions and human-perception sensitivity to manage resources.Visual–tactile movement should be perceived within 1 ms; biometric data also raises privacy concerns, including leakage and profiling.Source: A. Multi-Modal Communications · pp. 18–19

B. AI-Native Immersive Communications

  • AI-native immersive communications require lifecycle support for AI functions and policies that adapt prediction and resource management to network conditions and user behavior.The authors also identify poisoning of training data or crafted federated-learning models as a security concern that can bias results.Source: B. AI-Native Immersive Communications · p. 19

C. Time-Sensitive and Deterministic Networking

  • TSN and DetNet efforts mainly target layers 2 and 3 and industrial networks, so their current solutions may not transfer readily to all immersive use cases.Proposed directions include cross-layer delay minimization, joint prioritization of multimodal streams, and environment-aware, service-oriented management,,.Source: C. Time-Sensitive and Deterministic… · p. 20

D. QoE-Oriented Networking

  • Personalized QoE remains difficult because laboratory tests are costly and static, while objective assessments do not distinguish individual users.User digital twins could support personalized QoE modeling and resource management through timely, fine-grained user information, but preference data raises privacy concerns,,.Source: D. QoE-Oriented Networking · p. 20

E. Collaborative Multi-tier Computing

The paper identifies collaborative multi-tier computing and adaptable network architectures as directions for immersive communications. Its conclusion emphasizes that stringent requirements and computing-intensive, multimodal services call for substantial resources and new paradigms.

E. Collaborative Multi-tier Computing

  • Collaborative multi-tier computing can match immersive-communication tasks to servers by resource demands and manage context data for stateful tasks.Coordinating servers across tiers complicates context management, while trust among servers and users and user-credibility measurement remain open issues.Source: E. Collaborative Multi-tier Computing · pp. 20–21

F. New Network Architecture

  • Candidate architectures include heterogeneous multi-tier computing, digital twins with slicing and pervasive AI, O-RAN, and user-centric networking.Because none is designed specifically for immersive communications, customization and new designs remain open for investigation,,,,.Source: F. New Network Architecture · p. 21

VI. CONCLUSION

  • The survey reviews immersive-communication concepts, use cases, challenges, potential solutions, and future directions across XR, haptics, and holography.It presents their general procedures, network requirements, and recent developments in the context of a 6G vision.Source: §VI · p. 21
  • Multimodal and computing-intensive services compound stringent rate, delay, and reliability requirements, indicating a need for unprecedented communication and computing resources.The conclusion highlights AI-native communication, multi-tier computing, and user-centric networking as novel paradigms, and calls for research from physical-layer technologies to application-layer solutions.Source: §VI · p. 21
Loading 2303.05283v1…