PaperlayerView exact revision on arXiv

Source-linked AI summary

Scene Reconstruction Using Ubiquitous Wireless Signals via Diffusion Posterior Inference Over physics-Grounded Representations

Vahid Yazdnian, Yasaman Ghasempour

arXiv:2610.22120v1cs.LG

TL;DR

Wireless scene reconstruction is ill-posed because a channel measurement can be consistent with multiple scenes. The paper combines physics-grounded representations with joint diffusion posterior inference, and reports reconstruction of unseen scenes from communication signals while controlled comparisons support the importance of both stages.

  • Problem

    Wireless scene reconstruction is ill-posed: observations leave some occupancy unconstrained and can be explained by multiple scenes.

    Source: §2.2 · p. 3

  • Method

    A wireless foundation model encodes propagation-grounded scene information from each measurement, and a diffusion model jointly samples the occupancy posterior conditioned on multiple representations.

    Source: §3 · p. 4, §3.1 · p. 5

  • Results

    The framework reconstructs previously unseen building footprints from communication measurements, and controlled comparisons show that both physics-grounded representations and joint posterior inference matter.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The results demonstrate that communication-infrastructure measurements can support reconstruction of unseen scene structures without assumptions about structure number, shape, or placement.

    Source: §4.1 · p. 7, §5 · p. 9

Abstract

from arXiv · show

Knowing the layout of the physical world matters for many applications, yet the sensors we rely on for it (e.g., cameras and LiDAR) can fail in darkness, in adverse weather, or behind walls. Wireless signals do not face these limitations: every link between a base station and a user is shaped by the buildings it passes, reflects from, or is blocked by, leaving information about the surrounding scene embedded in the wireless channel measured at the endpoints. However, mapping channel measurements to the environment is a severely ill-posed problem: a measurement records only the structures that interact with the transmitted electromagnetic waves along its propagation paths, and even these observations can often be explained by multiple plausible scenes. We therefore formulate wireless scene reconstruction as posterior inference and develop a hierarchical framework with two complementary stages to approach this problem. A wireless foundation model, pretrained to respect propagation physics, transforms each channel measurement into a representation of the scene structure that shaped the observation. A diffusion model then jointly integrates representations from multiple measurements to model and sample the posterior over building occupancy, with consensus over samples producing the final reconstruction. We demonstrate that the framework can accurately recover building footprints from wireless channel measurements alone and show that its performance depends critically on both design components: physics-grounded representations expose scene-relevant information, while joint posterior inference coherently integrates complementary observations.

1 INTRODUCTION

Wireless channels encode scene geometry, but recovering a scene from them is structurally ambiguous. The paper treats reconstruction as posterior inference and combines physics-grounded representations with joint diffusion inference, demonstrating reconstruction from communication measurements and the importance of both stages.

1 INTRODUCTION

  • Prior RF reconstruction approaches use dedicated radar hardware, while deployed communication signals could support mapping through occlusion, darkness, or severe weather.Earlier wireless scene reconstruction is framed against radar approaches that can require additional hardware and movement through the environment.Source: §1 · pp. 1–2
  • A wireless link reveals only structures interacting with its propagation paths, and finite resolution and link-dependent interactions leave multiple scenes consistent with one measurement.Thus, a single measurement admits a family of possible scenes.Source: §1 · p. 2
  • The paper formulates reconstruction as posterior inference over occupancy without assumptions about scatterer number, shape, convexity, or location.Its stated contributions include bounds for a per-cell consensus estimator and a diffusion decoder that samples the occupancy posterior.Source: §1 · p. 2
  • The method first maps each channel measurement to a physics-grounded scene representation, then fuses multiple representations with a diffusion decoder and aggregates sampled scenes by per-cell consensus.The hierarchy separates learning propagation physics from inferring plausible scenes.Source: §1 · p. 2
  • Controlled comparisons show that independently aggregating per-observation estimates underperforms, while decoding directly from raw channels collapses.The evaluation uses about 700 held-out environments spanning diverse layouts and structural configurations.Source: §1 · p. 2

2 PRELIMINARIES AND PROBLEM FORMULATION

A wireless channel is a superposition of propagation paths shaped by scene geometry, but the resulting observations do not uniquely determine the scene. The formulation therefore distinguishes observable structure from unresolved occupancy and represents the solution as a posterior over scenes.

2.1 WIRELESS PROPAGATION

  • Each propagation path interacts with scene structures through direct travel, reflection, diffraction, or blockage, with its delays, angles, and gains determined by geometry.The receiver observes the paths in superposition as a MIMO channel.Source: §2.1 · p. 3
  • Only structures that affect a path enter the channel; structures outside an observation’s propagation-relevant set leave that measurement unchanged.The forward operator maps scene and link geometry to the observed channel.Source: §2.1 · p. 3

2.2 THE INVERSE PROBLEM

  • The target is a two-dimensional binary occupancy grid inferred from deployed-network channel measurements, without parametric assumptions about structure number, shape, convexity, or location.The observable set is the union of regions influencing at least one measurement.Source: §2.2 · p. 3
  • The inverse problem is non-unique because cells outside the observable set are unconstrained, finite bandwidth and aperture limit resolution, and structures can play indistinguishable roles across links.More observations can narrow the possible-scene family, but these ambiguities persist.Source: §2.2 · p. 3

2.3 A PROBABILISTIC FORMULATION

  • Because observations do not determine one scene, the solution is the posterior over scenes given the observations rather than a single uniquely recovered layout.The posterior separates the scene information extracted from data from the remaining uncertainty supplied by the prior.Source: §2.3 · p. 3
  • Measurements can change occupancy marginals in the observable region, while outside it the marginals remain prior-conditional and retain uncertainty.A deterministic regressor cannot represent this distinction and may make confident predictions where observations provide no constraint.Source: §2.3 · p. 4

3 DESIGN AND METHODOLOGY

The method separates learning how propagation physics shapes channels from inferring scenes consistent with multiple observations. PHOENIX produces physics-grounded representations, and a conditional diffusion model samples the occupancy posterior for consensus reconstruction.

3 DESIGN AND METHODOLOGY

  • The hierarchy uses a pretrained multimodal foundation model to encode each observation, then conditions a diffusion model on representations from multiple links to infer occupancy.This separates shared propagation-physics learning from scene inference for a given observation set.Source: §3 · p. 4

3.1 PHYSICS-GROUNDED REPRESENTATION LEARNING

  • PHOENIX uses a multimodal transformer whose [PHYSC] token aggregates channel, position, and spatial-temporal signal information into a shared representation.Inputs include channel tokens, scene-map patches, and a receiver-location token.Source: §3.1 · pp. 4–5
  • PHOENIX pretraining combines bidirectional reconstruction of masked channel and scene information with physics-guided distillation that aligns its representation to propagation characteristics and relevant scene structure.Scene masking is restricted to propagation-relevant patches, and distillation uses spatial-spectrum information and scene representations.Source: §3.1 · p. 5

3.2 DIFFUSION POSTERIOR OVER SCENES

  • The diffusion stage models the occupancy posterior directly from the set of physics-grounded representations.Those representations summarize what individual wireless observations reveal about the environment.Source: §3.2 · p. 5, §3.1 · p. 5
  • The binary occupancy grid is relaxed to analog bits, corrupted with Gaussian noise, and denoised through a noise-prediction objective.The training procedure samples a timestep and noise, constructs a noisy scene, and trains the denoiser to recover the injected noise.Source: §3.2 · p. 6
  • A Transformer denoiser uses grid cells as queries and observation embeddings as keys and values, allowing variable numbers of links to condition cell predictions.Cross-attention lets each cell draw on observations that encode evidence about that part of the scene.Source: §3.2 · p. 6, Figure 3 · p. 6
  • At inference, independent posterior samples are binarized, averaged into per-cell marginals, and thresholded to produce the consensus occupancy map.The estimated marginals concentrate around the learned posterior marginals, and the consensus approaches a per-cell Bayes-optimal decision under that posterior.Source: §3.2 · p. 6
  • Consensus retains structure consistently supported across posterior samples while suppressing structure arising from unresolved uncertainty.When the posterior is sufficiently close to the underlying distribution, consensus recovers the true scene on the observable set; cells outside it remain governed by the conditional prior.Source: §3.2 · pp. 6–7

4 EXPERIMENTS

The experiments test reconstruction across diverse held-out environments and compare physics-grounded representations and joint posterior inference with alternative designs. Results cover footprint reconstruction, the interpretability of per-measurement beliefs, performance across scene density, and how performance changes with measurement count.

4.1 MAIN RESULTS

  • Across approximately 700 held-out environments, 500 measurements from one base station recover building footprints, but ULA measurements leave building height severely underspecified.The reconstruction is a two-dimensional occupancy grid; a three-dimensional extension using an array that resolves elevation is left for future work.Source: §4.1 · p. 7, §4 · p. 7
  • Across all metrics, the framework achieves the strongest performance, and its proposed design improves over Marginal Prob. + FM by 22.1% in balanced accuracy and 67.0% in F1 score.Source: §4.1 · p. 8
  • Joint posterior inference outperforms fused marginals, with their median IoU differing by 3% relative and the gap reaching 16% at the 90th percentile.The advantage is concentrated in the upper tail of the per-scene distribution.Source: §4.1 · p. 8

4.2 PHYSICAL INTERPRETABILITY OF THE LEARNED BELIEFS

  • Per-measurement occupancy beliefs recover geometry along direct and reflected paths while remaining near one-half in regions the measurement does not inform.The beliefs capture reflecting structures and, when the direct path is blocked, the obstructing component.Source: §4.2 · p. 8, Figure 5 · p. 8

4.3 PERFORMANCE ACROSS SCENE DENSITY

  • Diffusion + FM performs best across sparse, moderate, and dense scenes, with its advantage over Marginal Prob. + FM growing as density increases.Its balanced-accuracy advantage grows from 2.9% to 13.2%, and its IoU advantage from 9.0% to 42.7%.Source: §4.3 · p. 9, Table 2 · p. 9

4.4 EFFECT OF THE NUMBER OF WIRELESS MEASUREMENTS

  • Diffusion + FM and marginal-probability variants improve and then saturate as measurements increase, while Diffusion + Raw Meas. remains near chance.The joint-posterior model increasingly surpasses Marginal Prob. + FM as complementary measurements accumulate.Source: §4.4 · p. 9, Figure 6 · p. 9

5 CONCLUSION

The paper concludes that physics-grounded representations paired with joint generative posterior inference can recover unseen scene structures from wireless channels. Its appendices detail the path-based forward model and explain why occupancy remains ambiguous outside observed or resolvable scene structure.

5 CONCLUSION

  • The framework combines a wireless foundation model with a diffusion model that jointly integrates observations and uses per-cell consensus to recover building footprints.The authors report recovery of previously unseen structures from communication signals and identify existing wireless infrastructure as a potential environmental-perception resource.Source: §5 · p. 9

B ILL-POSEDNESS OF THE INVERSE PROBLEM

  • The inverse problem has residual ambiguity even after observations constrain the regions their propagation paths reach.The conclusion frames the method as addressing this ill-posed reconstruction problem rather than eliminating all scene ambiguity.Source: §5 · p. 9, §B.1 · p. 13

A FORWARD MODEL DETAILS

  • Ray-traced direct, reflected, and diffracted paths determine the channel, while the propagation-relevant set records traversed and structure-interaction cells.Path gains aggregate free-space loss and reflection or diffraction coefficients.Source: A FORWARD MODEL DETAILS · p. 12

B.1 EXACT INVARIANCE OUTSIDE THE PROPAGATION-RELEVANT SET

  • At least 2^|A_c| scenes can match an observation because cells outside its propagation-relevant set do not affect that link.Across observations, the lower bound is 2^|A_c|, so residual non-uniqueness grows exponentially with untouched cells.Source: §B.1 · p. 13

B.2 FINITE RESOLUTION

  • Finite bandwidth and antenna aperture limit delay and angular resolution, leaving scenes with sufficiently similar path parameters indistinguishable.The stated resolutions are Δτ = 1/B and approximately Δθ = 2/N_tx near broadside.Source: §B.2 · p. 13

B.3 PROPAGATION-EQUIVALENT SCENES

Scenes can be indistinguishable when they induce identical path parameters, and multiple geometric mechanisms create such ambiguity. Independent observations sharpen the posterior by intersecting consistent-scene families, but do not necessarily determine every cell.

B.3 PROPAGATION-EQUIVALENT SCENES

  • Scenes inducing identical path parameters produce identical channels and cannot be distinguished, regardless of bandwidth or aperture.Distinct interaction sequences can alias to the same delay and departure and arrival angles.Source: §B.3 · p. 13
  • A cell's role depends on link geometry, and distinct occupancy patterns can produce the same role and propagation behavior.Roles include blocking, reflecting, free-space traversal, or no interaction.Source: §B.3 · p. 13
  • A two-dimensional occupancy grid omits building height, so a direct ray passing above an occupied cell can leave its occupancy unresolved.The unobstructed channel can be consistent with either free space or a structure the ray clears.Source: §B.3 · p. 13

B.4 COLLECTIVE OBSERVATIONS

  • Under conditional independence, additional observations sharpen the posterior by intersecting consistent-scene families, but residual ambiguities remain.Cells outside the aggregate propagation-relevant set retain prior-conditional uncertainty, while ambiguities within that set can also persist.Source: §B.4 · p. 14

C FOUNDATION MODEL ARCHITECTURE AND TOKENIZATION

The foundation model tokenizes channel, scene, and location inputs with physical positional information, then fuses them in a transformer. Its pretraining objectives target channel and scene structure relevant to propagation; the diffusion decoder samples occupancy scenes conditioned on the resulting representations.

C FOUNDATION MODEL ARCHITECTURE AND TOKENIZATION

  • Channel tokens group neighboring subcarriers while retaining transmit- and receive-antenna resolution; scene patches and receiver location form separate modalities.Sinusoidal physical-position encodings and modality embeddings identify token origin before joint transformer processing.Source: C FOUNDATION MODEL ARCHITECTURE AND… · p. 14

D PRETRAINING OBJECTIVE IN FULL

  • Pretraining combines channel and location reconstruction, propagation-relevant scene-patch prediction, spatial-spectrum matching, and contrastive alignment.Scene masking is restricted to propagation-relevant patches, avoiding training the model to infer regions that left no trace in the channel.Source: D PRETRAINING OBJECTIVE IN FULL · p. 15

E DIFFUSION DECODER DETAILS

  • The diffusion denoiser uses timestep-modulated self-attention and cross-attention to conditioning inputs, with conditioning dropout during training.Sampling starts from Gaussian noise; independent trajectories are binarized by sign at the final step for the stated concentration results.Source: E DIFFUSION DECODER DETAILS · pp. 15–16

F PROOFS OF THE CONSENSUS PROPOSITIONS

The propositions characterize consensus over posterior samples: it approaches per-cell Bayes decisions under the learned posterior and, under stated assumptions, recovers observable true scene values as informative observations accumulate. Unobservable cells retain prior-conditional uncertainty, and the practical rate of recovery depends on observations that distinguish competing scenes.

F PROOFS OF THE CONSENSUS PROPOSITIONS

F.2 PROOF OF PROPOSITION 4

  • For cells outside the observable set, measurements cannot reduce the prior-conditional variance; for observable cells, consensus recovers the true occupancy under the proposition's assumptions.The stated conditions include a well-specified posterior, sufficient conditioning representations, forward invariance, and conditional independence.Source: F PROOFS OF THE CONSENSUS PROPOSITIONS · p. 16, §F.2 · p. 17

F.1 PROOF OF PROPOSITION 3

  • Cells with posterior probability near 1/2 need more samples for stable consensus and also show greater sample disagreement.Sampling extracts a decision already encoded in the learned posterior; it adds no information of its own.Source: §F.1 · p. 16

F.2 PROOF OF PROPOSITION 4

  • Recovery at a cell improves with the number of observations that separate the true scene from alternatives differing at that cell.The error decays in the separating-observation count, not simply the number of observations covering the cell.Source: §F.2 · p. 17
  • Occupancy-grid coarseness leaves heights, materials, and sub-cell geometry unspecified, limiting attainable accuracy even as observations accumulate.The passage also notes that real propagation leaks weak energy off dominant paths, making the boundary of the observable set soft in practice.Source: §F.2 · p. 18

G DATASET AND TRAINING DETAILS

The appendix documents the simulated dataset and training setup, then specifies the marginal-belief and raw-measurement baselines used to assess the hierarchy.

G DATASET AND TRAINING DETAILS

  • The dataset contains over 30 million simulated channel samples from more than 11,000 outdoor environments, with occupancy represented on a 20 × 20 grid.The environments span 200 cities; each covers 200 m × 200 m.Source: G DATASET AND TRAINING DETAILS · p. 18
  • Training uses over 11,000 simulated environments, a frozen foundation-model encoder for cached embeddings, and a diffusion decoder evaluated on 671 held-out environments with nested observation sweeps.The diffusion decoder is trained with a uniformly redrawn observation count from 100 to 500; evaluation reuses each scene’s observations from the K = 1000 sweep for smaller K.Source: G DATASET AND TRAINING DETAILS · p. 18

H MARGINAL PROB. + FM

  • Marginal Prob. + FM predicts occupancy beliefs from each channel and location, then fuses confident per-observation beliefs using confidence and geometric weights.Its loss trains unconstrained cells toward 1/2; propagation-relevant sets are used for training the loss, not required at inference.Source: H MARGINAL PROB. + FM · pp. 18–19

I BASELINES

  • The raw-measurement baselines replace foundation-model embeddings with learned raw-channel conditioning or a from-scratch encoder, while retaining the corresponding decoder or fusion setup.Marginal Prob. + Raw Meas. uses the same fusion rule as Marginal Prob. + FM, but its loss drives even unconstrained cells toward hard labels, making the confidence gate ineffective.Source: I BASELINES · pp. 19–20

J EVALUATION METRICS

The evaluation reports balanced accuracy, F1, and IoU over the full occupancy grid, with each metric capturing a different aspect of reconstruction quality.

J EVALUATION METRICS

  • All three metrics are computed per scene over the full N × N grid and averaged across the 671 test scenes.Occupied cells are the positive class; TP, FP, TN, and FN define the reported outcomes.Source: J EVALUATION METRICS · p. 20
  • Balanced accuracy weights occupied and free cells equally, so constant or random predictions score 50% regardless of occupancy prevalence.The paper uses it as an indicator of chance-level behavior.Source: J EVALUATION METRICS · p. 20
  • F1 and IoU focus on the occupied class and are stricter footprint-recovery measures because correctly predicted free space is insensitive to them.For a single scene, IoU = F1/(2−F1); they can differ after averaging across scenes.Source: J EVALUATION METRICS · p. 20

K PHYSICAL INTERPRETATION OF PRETRAINED REPRESENTATIONS

The representation-space analysis compares [PHYSC] embeddings with raw channels and finds clearer structure aligned with propagation-relevant properties and environment identity.

K PHYSICAL INTERPRETATION OF PRETRAINED REPRESENTATIONS

  • Under user-position, propagation-direction, and scene-identity colorings, [PHYSC] embeddings show markedly clearer structure than raw channels.Figure 7 presents the same two-dimensional projections under all three colorings.Source: K PHYSICAL INTERPRETATION OF… · p. 21, Figure 7 · p. 21
  • Position and propagation-direction separation indicate that the representation resolves link endpoints and disentangles angular components that determine which structures links interact with.These properties are relevant to placing evidence in the scene and distinguishing propagation components mixed in raw channels.Source: K PHYSICAL INTERPRETATION OF… · p. 21
  • Observations from the same environment cluster nearby despite different users and channels, exposing environment information useful for conditioning the scene posterior.The raw channels contain at least as much information as their embeddings, but shared-environment observations are not neighbors in raw space, leaving the decoder to learn the mapping to scene structure.Source: K PHYSICAL INTERPRETATION OF… · p. 21

L AVAILABILITY OF POSITION INFORMATION

Position labels anchor propagation evidence to scene locations; reconstruction generally improves as more measurements have positions, with the raw-measurement diffusion variant remaining near chance.

L AVAILABILITY OF POSITION INFORMATION

  • Diffusion + FM rises from 0.56 balanced accuracy at 10% position availability to 0.76 at 100%, while Diffusion + Raw Meas. stays near 0.50.Figure 8 reports mean reconstruction accuracy as the fraction of measurements with position information varies.Source: L AVAILABILITY OF POSITION INFORMATION · p. 21, Figure 8 · p. 21
  • User positions map propagation evidence to scene cells, while Diffusion + Raw Meas.’s near-constant sweep indicates its poor reconstruction is not primarily due to insufficient position information.Position availability changes how much spatially anchored evidence is available for reconstruction.Source: L AVAILABILITY OF POSITION INFORMATION · pp. 21–22
  • Performance largely saturates once roughly half of measurements have position information, and the marginal and diffusion formulations differ in how they use sparse versus richer anchored evidence.Marginal Prob. + FM initially outperforms Diffusion + FM at low availability, but their ordering reverses as more position-labeled observations become available.Source: L AVAILABILITY OF POSITION INFORMATION · p. 22

M ROBUSTNESS TO NOISY POSITION INFORMATION

With user positions perturbed at test time, both designed pipelines remain far above the baselines, but conflicting mislocalized evidence can suppress recovered footprints. Figures 9 and 10 compare all models’ reconstructions with ground-truth scenes.

M ROBUSTNESS TO NOISY POSITION INFORMATION

  • The test uses independently perturbed user positions with σ = 5 m per horizontal axis and K = 500 observations per scene.All models are trained on exact positions; the radial error has a median of about 6.3 m and a 95th percentile of about 12.2 m, a range described as representative of consumer GNSS in outdoor urban settings.Source: M ROBUSTNESS TO NOISY POSITION… · p. 22
  • Both designed pipelines remain far above the baselines under noisy user positions, while conflicting mislocalized evidence can suppress recovered footprints.The diffusion decoder has never seen mislocalized evidence during training; disagreement between posterior samples moves affected-cell marginals toward 1/2, mainly removing true positives.Source: M ROBUSTNESS TO NOISY POSITION… · p. 22, Table 3 · p. 22
  • The authors attribute Diffusion + FM’s sensitivity to train–test mismatch and leave training with perturbed or missing-position embeddings for future work.They propose keeping clean occupancy targets while training the decoder on embeddings computed from perturbed or missing positions.Source: M ROBUSTNESS TO NOISY POSITION… · p. 22

N VISUAL DEMONSTRATION OF SCENE RECONSTRUCTION ALL MODELS

Related papers

Loading 2610.22120v1…