PaperlayerView exact revision on arXiv

Source-linked AI summary

Event-Native Immersive Communication for 6G: From Multimodal Streams to Event Beliefs

Yiyang Zhu, Li Wei, Chau Yuen

arXiv:2610.12270v1cs.ITeess.SPmath.IT

TL;DR

Immersive communication lacks a clear target for deciding what multimodal information to preserve, and preserving separate stream or task targets does not necessarily preserve the event perceived or acted upon. The paper makes service-relevant events and transceiver beliefs its communication framework and defines EBD to evaluate event-level fidelity. It derives reliability characterizations for a Gaussian, mode-separable Rayleigh setting, including an optimistic hard-ceiling benchmark under matched parameters.

  • Problem

    Existing approaches optimize rendered features, feedback loops, or task-specific utilities, leaving unresolved whether separately preserving these targets preserves the event perceived or acted upon.

    Source: A. Related Works and Limitations · p. 2

  • Method

    The framework makes the physical event the communication object, represents agent information with event beliefs, and defines EBD as communication-induced loss of event-dependent decision capability.

    Source: §VI · p. 12

  • Results

    Under matched bi=ci, the hard-ceiling model gives DH(s) ≤ DG(s; λ) and κδ,H ≤ κδ,G(λ), providing an optimistic but tractable benchmark.

    Source: E. Resource Coupling for Event-Native… · p. 9

  • Takeaways & Limitations

    The findings support formulating immersive communication around preserving physical-event beliefs rather than reconstructing individual modality streams.

    Source: §VI · p. 12

    The supplied scope leaves unresolved whether separately preserving rendered-feature, feedback-loop, or task-utility targets preserves the joint event.

    Source: A. Related Works and Limitations · p. 2

Abstract

from arXiv · show

The development of multimodal sensing capabilities is crucial for realizing immersive communications and enabling rich, natural remote interactions in future sixth-generation~(6G) communications. However, transmitting high-dimensional sensory streams places substantial pressure on wireless resources and leaves a fundamental question unresolved: what information about the underlying physical process should be preserved across wireless delivery? To address this issue, we establish an event-native immersive-communication framework in which a service-relevant physical event defines the communication object, while event beliefs represent the information available to transceiver agents before and after wireless delivery. Based on this event-native framework, we further propose event-belief deficiency (EBD) to evaluate quality of experience (QoE) in immersive communications, which differs from the conventional normalized mean-squared error (NMSE), as NMSE can be insensitive to physically distinct procedures whose stream distortions lie in a similar range. For Gaussian event beliefs with mode-separable delivery over Rayleigh fading, we derive the log-det EBD, its induced received signal-to-noise ratio (SNR) threshold, the exact outage probability, and the required transmit SNR. Simulation results with a physics-driven source show that stream NMSE can remain nearly unchanged across physically distinct event procedures, whereas log-det EBD reliably reflects their event-level uncertainty. Relative to a mean-squared-error (MSE)-oriented interface, event-native delivery reduces normalized log-det EBD by up to $63.6\%$ and lowers the required average received SNR by $3.47$~dB under the evaluated outage constraint, substantiating the effectiveness of the proposed framework.

I. INTRODUCTION

The paper argues that immersive communication should preserve service-relevant physical events rather than sensory streams, and introduces event beliefs and EBD as its framework and fidelity criterion.

I. INTRODUCTION

  • The paper identifies the central problem as choosing what multimodal information to preserve when wireless constraints make stream delivery costly and signal distortion an unreliable proxy for event understanding.Small signal errors can alter temporal evolution or physical consistency, while large samplewise deviations may not affect event-level understanding.Source: §I · p. 1

A. Related Works and Limitations

  • Existing media, haptic, and task-oriented approaches target rendered features, feedback-loop operation, or task-specific utility, leaving event-level preservation across coupled modalities unresolved.Their criteria remain tied to selected media, haptic-channel metrics, or case-specific objectives.Source: A. Related Works and Limitations · pp. 1–2

B. Main Contributions

  • The framework makes a service-relevant physical event the communication object and uses transceiver beliefs to represent available event information before and after wireless delivery.It also treats conventional source transmission and stream reconstruction as special cases.Source: B. Main Contributions · p. 2
  • EBD measures communication-induced distortion of event-dependent decision capability relative to the sensing-supported agent belief; for Gaussian beliefs and mode-separable Rayleigh delivery, the paper derives LD-EBD and reliability and SNR characterizations.The formulation also relates event-mode resources to reliability thresholds through delivered precision.Source: B. Main Contributions · p. 2
  • Up to 63.6% lower normalized LD-EBD and 3.47 dB lower required average received SNR were achieved by event-native delivery versus an MSE-oriented interface under the evaluated LD-EBD outage constraint.In the physics-based evaluation, stream NMSE was nearly insensitive to physically distinct event procedures, while LD-EBD reflected their event-level uncertainty.Source: B. Main Contributions · p. 2

A. System Overview

  • The system model has a transmitter-side agent infer an event from synchronized heterogeneous sensor records and a receiver-side agent infer the same event from wireless evidence.The receiver uses its posterior belief for subsequent inference or decision.Source: A. System Overview · p. 3

B. Physical Process and Multimodal Sensing

  • The physical-process model represents a spatiotemporal field over a region and observation window, with synchronized sensors providing complementary measurements whose joint law captures cross-modal dependence.The sensors observe the same physical evolution, and the resulting records form the transmitter-side evidence about the episode.Source: B. Physical Process and Multimodal… · p. 3

C. Service-Relevant Event and Transmitter-Side Inference

A service-relevant event groups physical episodes that are indistinguishable under the declared interpretation targets. The transmitter infers that event from multimodal sensing and uses its posterior belief as a sufficient information state for event-dependent decisions.

C. Service-Relevant Event and Transmitter-Side Inference

  • The event is an equivalence class of physical episodes that induce identical conditional laws under all declared interpretations.Episodes in one class may differ in trajectories and sensor records; the resulting event space is the coarsest deterministic state preserving those laws.Source: C. Service-Relevant Event and… · p. 3
  • The event specifies aspects of an episode relevant to an interpretation, while within-event variability is marginalized to obtain the event-conditioned sensing likelihood.The stated inference uses conditional independence of interpretation and sensing residuals given the event.Source: C. Service-Relevant Event and… · pp. 3–4
  • Combining the event prior and sensing likelihood gives the transmitter-side posterior event belief, which captures sensor-supported event information and remaining uncertainty.The belief determines the posterior predictive law of the interpretation target.Source: C. Service-Relevant Event and… · p. 4
  • For any Bayesian decision problem with loss on the interpretation target, an optimal decision rule based on the sensor records can be expressed as a function of the transmitter-side belief.This sufficiency result justifies using the belief as the transmitter's information state for event-dependent encoding.Source: C. Service-Relevant Event and… · p. 4

D. Event-Native Encoding and Wireless Transmission

The transmitter forms an event representation from its posterior event belief and encodes it for wireless delivery. The receiver infers its event belief from the received signal and channel state using the end-to-end observation model.

D. Event-Native Encoding and Wireless Transmission

  • The transmitter generates an event representation S from its posterior event belief, then maps S to a length-n channel input U.The intermediate representation separates event representation from channel encoding.Source: D. Event-Native Encoding and Wireless… · p. 4
  • With deterministic sensing and a service requiring full-record reconstruction, the event-native model reduces to stream-oriented transmission.In this special case, the event map preserves all sensor-domain distinctions, so Q = X1:M up to bijective relabeling.Source: D. Event-Native Encoding and Wireless… · p. 4
  • The transmitter sends U over a block-fading channel whose coefficient remains constant for n channel uses; the receiver's wireless evidence is (Y, H).Given the channel input and channel state, the received block is independent of upstream variables.Source: D. Event-Native Encoding and Wireless… · p. 4

E. Receiver-Side Event Inference

  • The receiver forms a posterior event belief from the wireless evidence through the end-to-end observation model, rather than reconstructing it from the transmitter's belief.The transmitter and receiver beliefs concern the same event space but are conditioned on different evidence.Source: E. Receiver-Side Event Inference · pp. 4–5

III. EVENT-BELIEF DEFICIENCY: FROM DECISION RISK TO WIRELESS RELIABILITY

The section argues that immersive communication should be evaluated by the event-dependent decisions supported by information before and after delivery, rather than by stream distortion alone. It establishes the transmitter-side belief as the communication reference and orders the resulting decision risks.

A. From Stream Distortion to Event-Dependent Decision Risk

  • Event-dependent decision risk is the minimum achievable task loss given an information state, over admissible decision rules.For a fixed task, the risk is independent of a particular inference implementation.Source: A. From Stream Distortion to… · p. 5
  • Sensor-domain reconstruction quality does not generally determine event-dependent decision capability.There can be two receiver information states with the same minimum stream-reconstruction mean-squared error but different minimum squared-error risks for the same event-dependent quantity.Source: A. From Stream Distortion to… · p. 5

B. Communication Reference and Risk Decomposition

  • The transmitter-side belief is the pre-communication reference, excluding uncertainty that remains because of multimodal sensing.The prior, transmitter-reference, and receiver risks respectively reflect no event-window observations, sensing before transmission, and the full communication chain.Source: B. Communication Reference and Risk… · p. 5
  • For every finite-risk event-dependent task, Dref ≤ Drx ≤ D0.Wireless evidence is generated from the transmitter belief through a stochastic kernel, so the receiver can simulate its observation from that belief; the receiver can also ignore its observation.Source: B. Communication Reference and Risk… · pp. 5–6
  • The risk gap D0 − Dref is the task-specific decision improvement from sensing, while Drx − Dref is the capability lost through representation and wireless delivery.The receiver risk is evaluated after the complete communication chain; Dref may remain positive because of sensing noise, limited coverage, and unresolved physical variability.Source: B. Communication Reference and Risk… · pp. 5–6

C. Event-Belief Deficiency

Event-belief deficiency measures the communication-induced loss in event-dependent decision capability relative to the information available at the transmitter. The section describes its normalized form and its log-loss interpretation.

C. Event-Belief Deficiency

  • Event-belief deficiency is the increase in minimum decision risk from the transmitter belief to the receiver evidence for a given task and communication mapping.It isolates loss introduced by representation, encoding, and the wireless channel relative to the event information available at the transmitter.Source: C. Event-Belief Deficiency · p. 6
  • Normalized EBD is the fraction of sensing-supported decision value lost through communication, with endpoints for complete preservation and no decision gain over the prior.For a given task, EBD can be zero even when the transmitter and receiver beliefs differ realizationwise.Source: C. Event-Belief Deficiency · p. 6
  • Under log loss, EBD is the information about Z available in the transmitter belief but absent from the wireless evidence.The definitions characterize average degradation; reliability additionally controls large state-dependent deficiencies.Source: C. Event-Belief Deficiency · p. 6

D. Event-Belief Reliability and Resource Design

The section defines event-belief reliability by limiting state-dependent decision loss and formulates a resource-constrained design problem. It then outlines a Gaussian, mode-separable Rayleigh-fading analysis based on event uncertainty after sensing and delivery.

D. Event-Belief Reliability and Resource Design

  • State-dependent EBD compares receiver and transmitter-reference risks under the same operating-state realization.The conditioning variable stratifies service performance but does not augment the agents' information; its expectation recovers the corresponding average comparison.Source: D. Event-Belief Reliability and… · p. 6
  • Event-belief reliability requires the probability that state-dependent deficiency exceeds tolerance δ to be no greater than outage probability ϵ.The outage criterion combines the task, sensing uncertainty, communication design, and operating conditions.Source: D. Event-Belief Reliability and… · p. 6
  • The reliability requirement yields a resource-constrained design problem over admissible length-n mappings satisfying the power constraint.The framework jointly designs the physics-informed mapping and resource allocation.Source: D. Event-Belief Reliability and… · p. 7

IV. PERFORMANCE ANALYSIS OF EVENT-BELIEF DELIVERY OVER RAYLEIGH FADING

  • For Gaussian event beliefs with mode-separable delivery over Rayleigh fading, the analysis derives the LD-EBD-induced received-SNR threshold, outage probability, and required transmit SNR.It also relates the threshold to radio-resource allocation through delivered event-mode precisions.Source: §IV · p. 7

A. Gaussian Event Model and Reference Covariances

  • In the Gaussian specialization, the posterior covariance represents residual event uncertainty after multimodal sensing, and communication degradation is quantified by covariance inflation relative to the transmitter reference.The analysis distinguishes prior-only uncertainty from residual uncertainty after sensing but before wireless delivery.Source: A. Gaussian Event Model and Reference… · p. 7

B. Mode-Separable Belief Delivery

The delivery model assumes coherent block-Rayleigh fading and represents prior, transmitter, and receiver beliefs through covariances separable across a common set of event modes. Delivered precision in each mode grows with received SNR and is bounded by the sensing-supported precision.

B. Mode-Separable Belief Delivery

  • The channel is coherent block-Rayleigh fading: its gain is constant within each block, and the instantaneous received SNR is Γ = ρG with exponentially distributed G of mean β.The receiver belief is Gaussian at a realized received SNR.Source: B. Mode-Separable Belief Delivery · p. 7
  • Prior, transmitter-reference, and receiver covariance matrices are simultaneously diagonalizable in a fixed orthonormal basis whose coordinates are the event modes.Source: B. Mode-Separable Belief Delivery · p. 7
  • In each event mode, prior precision is a_i, sensing adds b_i,ref at the transmitter, and γ_i(s) is the portion of that increment retained at the receiver.The common task-relevant basis can arise when sensing, representation, and delivery mappings are aligned.Source: B. Mode-Separable Belief Delivery · p. 7
  • Under the adopted hard-ceiling precision law, delivered precision increases linearly with received SNR and saturates at the sensing-limited ceiling b_i,ref.Source: B. Mode-Separable Belief Delivery · p. 7

C. Log-Det EBD and Received-SNR Threshold

For Gaussian beliefs, the paper expresses communication-induced event-belief loss through posterior covariance determinants and decomposes it across event modes. A target on this loss defines a received-SNR threshold when the loss decreases continuously and strictly over the relevant range.

C. Log-Det EBD and Received-SNR Threshold

  • At received SNR s, the Gaussian log-loss EBD is the difference between receiver and transmitter posterior-uncertainty log determinants.The common Gaussian entropy term cancels in the difference, and the measure is in nats.Source: C. Log-Det EBD and Received-SNR… · p. 8
  • With mode-separable covariances, log-det EBD decomposes into nonnegative residual information-loss terms, one for each event mode.Source: C. Log-Det EBD and Received-SNR… · p. 8
  • The maximum LD-EBD occurs at zero received SNR.Source: C. Log-Det EBD and Received-SNR… · p. 8
  • For a feasible target, the EBD-induced threshold κδ is the unique received SNR at which the decreasing response meets that target.The response is assumed continuous and strictly decreasing where it is positive.Source: C. Log-Det EBD and Received-SNR… · p. 8
  • Under the hard-ceiling model, κδ,H can be computed by one-dimensional bisection because the response is continuous and strictly decreasing over the relevant interval.The threshold is independent of the fading distribution and depends on the Gaussian event and sensing parameters and delivery design.Source: C. Log-Det EBD and Received-SNR… · p. 8

D. Rayleigh Outage and Link-Budget Scaling

For Rayleigh fading, the event-belief threshold maps to outage and required transmit SNR; at high SNR, the outage has first-order diversity. The analysis also relates resource allocation across event modes to delivered precision and the resulting EBD threshold.

D. Rayleigh Outage and Link-Budget Scaling

E. Resource Coupling for Event-Native Delivery

  • The resource-coupled scheme assigns channel uses and power to active event modes and represents their transmitted information through equivalent scalar observations.The resulting communication-induced precision slope determines the delivered precision increment.Source: E. Resource Coupling for Event-Native… · p. 9
  • When b_i = c_i, the hard-ceiling model has the same low-SNR slope and sensing-limited ceiling, and its threshold is no greater than the Gaussian scheme's for a common feasible target.The hard-ceiling model is an optimistic tractable benchmark; the Gaussian scheme exposes dependence on radio-resource allocation.Source: E. Resource Coupling for Event-Native… · p. 9
  • The threshold characterization, combined with the outage result, provides a basis for target-dependent event-native radio-resource allocation.Source: E. Resource Coupling for Event-Native… · p. 9

V. NUMERICAL RESULTS

The numerical evaluation uses a physics-based fluid-interaction source to assess event beliefs and compare source interfaces under matched transmission constraints. It calibrates a three-mode event model from simulated event–observation pairs.

V. NUMERICAL RESULTS

  • The evaluation tests whether stream distortion tracks receiver knowledge of the event, compares event-native with stream-oriented interfaces under matched transmission constraints, and examines outage predictions and SNR requirements.Source: §V · p. 9

A. Physics-Based Environment and Belief Calibration

  • A moving contact jointly produces membrane deformation, pressure diffusion, vibration, and audio-related observations whose event-relevant components are linked through the latent physical event.The FIT example treats the modalities as observations of a shared physical cause rather than aligned by stream-domain energy.Source: A. Physics-Based Environment and… · p. 9, Figure 2 · p. 10
  • The nonlinear simulator generates 106 event–observation pairs, which calibrate a Gaussian observation model and a three-mode event basis for contact, pressure, and vibration information.Source: A. Physics-Based Environment and… · p. 10
  • Four interfaces are compared with identical transmitted dimension, blocklength, energy budget, and fading realizations: event-native, MSE-oriented, stream PCA, and uncoded analog.Performance uses normalized LD-EBD; an auxiliary normalized physical-state estimation error is included for interpretation.Source: A. Physics-Based Environment and… · p. 10

B. Metric Validity and Event-Native Delivery

Stream NMSE poorly tracks event-risk differences across physically distinct FIT procedures, while LD-EBD distinguishes them. Event-native delivery also improves event-belief fidelity and required-SNR performance under the evaluated outage constraint.

B. Metric Validity and Event-Native Delivery

  • Median NMSE changes by only 1.38% across physically distinct FIT procedures, while median LD-EBD changes by a factor of 17.32 and tracks event-risk ordering.Their Spearman correlations with the reference event-risk ordering are -0.035 for NMSE and 0.995 for LD-EBD.Source: B. Metric Validity and Event-Native… · p. 10, Figure 3 · p. 11
  • Event-native achieves the smallest normalized LD-EBD across the simulated SNR range, with its reduction versus MSE-oriented rising from 3.1% at low SNR to 63.6% at high SNR.The average reduction is 37.6%; as SNR rises, the interface's allocation of representation to sensing-supported event information becomes the dominant difference, while physical-state estimation remains competitive.Source: B. Metric Validity and Event-Native… · pp. 10–11, Figure 4 · p. 11

C. Reliability and Resource-Design Gains

VI. CONCLUSION

  • The conclusion motivates future work on nonlinear and learned event representations, interactive multi-agent systems, and experimental prototypes.Source: §VI · p. 12

APPENDIX A PROOF OF LEMMA 1

The proof establishes the Tx-A belief as sufficient for Bayesian decisions based on the event and shows that equal stream-reconstruction MSE need not imply equal event-dependent risk.

APPENDIX A PROOF OF LEMMA 1

  • The conditional law relevant to Bayesian decision problems based on the event is determined by the Tx-A belief, so an optimal rule based on sensor observations can be expressed as a function of that belief.Source: Appendix A · p. 12
  • The two Rx-A information states have the same minimum stream-reconstruction MSE, yet their event-dependent decision risks can differ.Source: Appendix A · pp. 12–13

APPENDIX C PROOF OF THEOREM 1

  • For 0 < δ < δmax, the received-SNR response D(s) is strictly decreasing while positive, and κδ is the unique solution to (65).The fading quantity is exponentially distributed with mean β.Source: Appendix C · p. 13
Loading 2610.12270v1…