PaperlayerView exact revision on arXiv

Source-linked AI summary

Array-Agnostic High-Order Ambisonics Encoding via Diffusion Posterior Sampling

Amit Milstein, Nir Shlezinger, Boaz Rafaely

arXiv:2610.39185v1eess.ASeess.SP

TL;DR

Practical Ambisonics recovery must address array-dependent distortions and limited spatial resolution, while encoding and upscaling are often treated separately. ADEUPS combines an array-independent diffusion prior with a physical acquisition model and order masking. Its objective and subjective evaluations show performance comparable to, and in many cases exceeding, the evaluated model-based and data-driven baselines.

  • Problem

    Practical Ambisonics acquisition depends on array geometry and transfer functions, while existing upscaling methods generally assume ideal low-order inputs and do not account for array artifacts.

    Source: §I · pp. 1–2

  • Method

    ADEUPS trains an array-independent diffusion prior with order masking and combines it at inference with a physical acquisition model and score blending to reconstruct HOA from microphone measurements.

    Source: §I · p. 2, C. Order-Masking Prior Training · p. 6, D. Discussion · p. 7

  • Results

    Objective and subjective evaluations show ADEUPS performs comparably to, and in many cases exceeds, array-agnostic model-based baselines and data-driven methods trained on specific array topologies.

    Source: §VI · p. 11

  • Takeaways & Limitations

    Evaluations support ADEUPS's zero-shot use across microphone geometries without exposure to specific array geometries during training.

    Source: §VI · p. 11

Abstract

from arXiv · show

Ambisonics provides a device-independent representation of three-dimensional sound fields; however, extracting these coefficients from practical microphone array recordings remains strongly dependent on array geometry, acoustic transfer functions, and spatial sampling resolution. This pipeline is typically divided into two distinct tasks: encoding, which entails reconstructing the physically resolvable spatial components, and upscaling, which requires generating higher-order coefficients. Model-based approaches often fail on irregular arrays due to ill-conditioned transfer matrices, whereas conventional neural encoders remain strictly tied to the specific acquisition setup seen during training. Furthermore, existing upscaling methods usually assume ideal low-order inputs, ignoring device-specific array artifacts. We present ADEUPS, a unified framework that couples an array-independent Ambisonic diffusion prior with an explicit physical acquisition model during inference, enabling zero-shot encoding and upscaling across diverse array configurations. We train our prior using an order-masking strategy and perform inference via score blending, which we demonstrate to be crucial for solving the underlying inverse problem. Through extensive numerical evaluations across various array geometries, we show that ADEUPS consistently outperforms task- and array-specific baselines. Finally, we conduct subjective listening tests to confirm the perceptual effectiveness of our approach.

I. INTRODUCTION

ADEUPS combines an array-independent Ambisonics prior with a physical acquisition model to reconstruct high-order representations from microphone measurements. The introduction motivates this unified approach by the distortions and limited spatial resolution of practical arrays, and describes its signal model and evaluation scope.

I. INTRODUCTION

  • Practical array recordings distort Ambisonics through geometry, transfer functions, limited spatial sampling, and ill-conditioned steering matrices, restricting reliably recoverable spherical-harmonic orders.The paper frames encoding and higher-order upscaling as coupled reconstruction tasks.Source: §I · p. 1
  • ADEUPS achieves state-of-the-art performance across diverse array configurations and reconstruction settings, surpassing array-specific and semi-agnostic baselines in numerical and listening evaluations.The framework targets both encoding and upscaling.Source: §I · p. 2
  • ADEUPS trains an array-independent prior on ideal HOA signals using order masking, then applies the physical acquisition model and score blending during inference.This separation supports zero-shot operation across microphone geometries and channel counts without retraining.Source: §I · p. 2
  • The framework treats encoding and order upscaling as a single inverse problem rather than separate mappings, avoiding error accumulation and bottlenecks in cascaded architectures..Source: §I · p. 2
  • The study evaluates diverse microphone geometries and counts, reconstruction orders, and both encoding and upscaling, using objective metrics and subjective listening tests..Source: §I · p. 2

A. Signal Model

  • Ambisonics represents a sound field through spherical harmonics weighted by frequency-dependent coefficients, while microphone recordings sample that field at array locations.The signal model describes each microphone pressure as a linear combination of Ambisonics coefficients.Source: A. Signal Model · p. 2

B. Problem Formulation

The formulation treats reconstruction of a desired Ambisonics order from microphone measurements as an inverse problem that includes both encoding and upscaling. It identifies physical and dimensional challenges, then positions conventional linear encoding and diffusion modeling as preliminaries to a unified solution.

B. Problem Formulation

  • The target reconstruction order can exceed the order reliably recovered by direct inversion, so the same reconstruction operator covers encoding and higher-order upscaling.Encoding recovers components supported by microphone observations; upscaling infers higher-order components insufficiently observed by the array.Source: B. Problem Formulation · p. 3
  • The inverse problem combines low-frequency noise amplification, direction- and channel-dependent errors, and frequency- and geometry-dependent spatial aliasing.These challenges arise from radial-function decay, irregular microphone arrangements, and insufficient spatial sampling.Source: B. Problem Formulation · p. 3
  • When the desired order exceeds the array's spatial dimensionality, such as when Q < (Np+1)^2, direct reconstruction is underdetermined and conventional encoding leaves higher orders unresolved..Source: B. Problem Formulation · p. 3
  • The stated objective is a unified spatial mapping across array geometries and microphone counts, addressing encoding degradations and high-order recovery within one inverse problem.Training is assumed to use ideal, array-agnostic order-Np Ambisonics coefficients.Source: B. Problem Formulation · p. 3

C. Preliminaries

  • The preliminaries review conventional linear encoding and basic generative diffusion modeling and sampling before introducing ADEUPS..Source: C. Preliminaries · p. 3

1) Linear Encoder:

  • Conventional linear encoding estimates coefficients from microphone measurements with a frequency-dependent linear operator and commonly uses a Tikhonov-regularized pseudoinverse.Its maximum order is bounded by Nenc = floor(sqrt(Q) - 1); stronger regularization suppresses unstable modes but increases model mismatch and spatial bias, while weaker regularization amplifies microphone noise.Source: 1) Linear Encoder · p. 3

2) Diffusion Models:

Diffusion models progressively corrupt data toward a tractable prior and generate samples by reversing that process. The paper reviews reverse-time stochastic or deterministic sampling and the denoiser-based score formulation used in EDMs.

2) Diffusion Models:

  • Diffusion models progressively corrupt clean samples from the data distribution toward a tractable prior, typically Gaussian.The forward process is parameterized by time and a noise schedule.Source: 2) Diffusion Models · p. 3
  • Generation reverses the corruption through a reverse-time stochastic differential equation, with the marginal score given by the gradient of the log density of the corrupted state.Sampling may also use the probability-flow ODE or an equivalent SDE solver.Source: 2) Diffusion Models · pp. 3–4
  • In the EDM formulation, the drift is removed and a neural denoiser is trained to estimate the score function.The denoiser is preconditioned across noise levels, with a skip connection to the data manifold as noise approaches zero.Source: 2) Diffusion Models · pp. 3–4

3) Diffusion Posterior Sampling:

Diffusion posterior sampling reconstructs an unknown signal by combining a diffusion prior with a likelihood score derived from its observations. ADEUPS formulates Ambisonics recovery in that framework, using an array-specific forward operator during inference.

3) Diffusion Posterior Sampling:

  • The posterior score combines an unconditional diffusion-prior score with a likelihood score that conditions reconstruction on the observations.The likelihood is approximated using an estimated clean-signal target because the exact likelihood at intermediate diffusion states is intractable.Source: 3) Diffusion Posterior Sampling · p. 4

III. ADEUPS

A. DPS Formulation for Ambisonics Reconstruction

  • Its forward operator maps an ideal compressed order-Np sample through STFT decompression, microphone sampling, Ambisonics encoding, and STFT compression.Modeling array-specific acquisition effects in this inference-time pipeline supports generalization across microphone geometries.Source: A. DPS Formulation for Ambisonics… · p. 4

B. Inference

ADEUPS inference blends an order-conditioned Ambisonics prior with an array-dependent measurement likelihood during posterior sampling. Generation and refinement receive different emphasis over the noise trajectory, and the final sample is returned as order-Np coefficients.

B. Inference

  • The posterior score adds the learned prior score to a likelihood score that enforces consistency with the array observation through the physical forward operator.The microphone configuration affects the posterior trajectory through the likelihood term, while the learned prior remains array-independent.Source: B. Inference · pp. 4–5
  • The denoiser generates higher-order components from trusted order-Nenc context and separately refines the complete order-Np representation.These generation and refinement modes are provided by the same order-conditioned denoiser.Source: B. Inference · p. 5
  • Early sampling blends generation and refinement, while sampling below σth focuses on full-order refinement.The relative contributions are controlled by λg, λr1, and λr2, and σth is set empirically.Source: B. Inference · p. 5
  • After the final sampling step, applying H^-1 yields the reconstructed order-Np Ambisonics coefficients.The procedure initializes around the encoded observation and uses a first-order Euler sampler with stochastic churn.Source: B. Inference · p. 5

C. Order-Masking Prior Training

ADEUPS trains an array-independent denoiser on ideal order-Np Ambisonics signals, masking complete orders to teach both higher-order completion and full-order refinement.

C. Order-Masking Prior Training

  • Training uses ideal order-Np Ambisonics signals only, without microphone signals, steering matrices, or array-dependent encoding operators.The resulting learned prior is independent of the acquisition configuration.Source: C. Order-Masking Prior Training · p. 6
  • Randomly masking complete Ambisonics orders varies how much spatial information is available to the denoiser during training.A sampled trusted order κ determines which orders remain available as context.Source: C. Order-Masking Prior Training · p. 6
  • The denoising target remains the complete clean signal, so κ<Np trains completion from lower-order context while κ=Np trains full-order denoising.These behaviors support the inference-time generation and refinement scores, respectively.Source: C. Order-Masking Prior Training · p. 6
  • Masked-channel preconditioning prevents pure-noise inputs from being directly propagated through the denoiser’s skip connection.Joint sampling of κ and diffusion noise level trains one denoiser for full-order modeling and completion; C=(Np+1)^2 is the channel count.Source: C. Order-Masking Prior Training · p. 6

D. Discussion

ADEUPS combines an ideal-Ambisonics prior with physical measurement consistency to address array-dependent acquisition effects and underdetermined higher-order recovery. Its scope depends on the assumed acquisition model and the chosen trusted-order rule.

D. Discussion

  • The posterior trajectory combines an ideal-Ambisonics prior with physical measurement consistency rather than relying solely on potentially ill-conditioned steering-matrix inversion.The discussion describes this as data-driven regularization for noise amplification and non-uniform sensitivity, while the forward operator accounts for array-dependent spatial aliasing.Source: D. Discussion · pp. 6–7
  • A single prior trained only on ideal Ambisonics signals can serve different microphone geometries and channel counts without retraining.Steering and encoding matrices enter through the test-time forward operator, while learned inter-order dependencies support components insufficiently determined by measurements.Source: D. Discussion · p. 7
  • The formulation sets hyperparameters on held-out validation data, but choosing Np<Neff omits higher-order contributions and creates forward-model mismatch.Np is specified by the reconstruction problem rather than treated as an algorithmic hyperparameter.Source: D. Discussion · p. 7
  • The current trusted order is determined by microphone count, although observability also depends on frequency, geometry, and steering-matrix conditioning.The likelihood assumes an accurately known steering matrix; confidence-based masks and modeling calibration uncertainty are proposed for future study.Source: D. Discussion · p. 7

IV. EMPIRICAL EVALUATION

The evaluation compares ADEUPS with Ambisonics reconstruction methods, then reports numerical results after describing the experimental setup and methodology.

IV. EMPIRICAL EVALUATION

  • ADEUPS is numerically evaluated against several Ambisonics reconstruction methods.The section is organized around experimental setup and datasets, methodology, and numerical results.Source: §IV · p. 7
  • The evaluation setup and datasets are outlined separately from the model configurations, baselines, and metrics.These are described before the numerical results.Source: §IV · p. 7
  • The section reports numerical results after detailing the evaluation setup and methodology.Source: §IV · p. 7

A. Experimental Setup

The experiments train on simulated acoustic impulse responses with VCTK speech, test on higher-order scenes and diverse arrays, and compare against model-based and learning-based baselines.

1) Training Data:

  • Training uses 20,000 simulated Ambisonics room impulse responses and randomly cropped VCTK speech segments with one or two active speakers.The simulated rooms span reverberation times of 0.1–0.5 s.Source: 1) Training Data · p. 7

2) Evaluation Data:

  • Evaluation uses 100 WSJ0 scenes at Neff = 15 across 13 array geometries, with 1,300 noisy microphone recordings.The arrays include Project Aria glasses and irregular arrays with four, five, or six microphones; noise is set to 50 dB SNR.Source: 2) Evaluation Data · p. 7

B. Methodology

  • The NCSN++M-based model has 31.1 million parameters and adds trusted-order conditioning with adaptive linear normalization.The fully connected input and output layers are resized for (Np+1)^2 channels.Source: B. Methodology · p. 7
  • Inference uses 90 sampling steps and 150 function evaluations, with order masking assigning equal probability to κ = Np and κ = 2 during training.The score weights are λg = λr1 = 0.7 and λr2 = 1.Source: B. Methodology · p. 7

1) Benchmarks:

  • Benchmarks include array-agnostic model-based methods and semi-agnostic or array-specific neural encoders and upscalers.The set includes a linear encoder, a parametric method, UNet-Encoder, Gen-A, Diff-Encoder, and Diff-Upscaler.Source: 1) Benchmarks · pp. 7–8
  • Diff-Encoder maps microphone recordings to Np-order Ambisonics, whereas Diff-Upscaler maps first-order Ambisonics to Np using arrays with four or six microphones in training.The model specifications are summarized in Table I.Source: 1) Benchmarks · p. 8

2) Evaluation Metrics:

ADEUPS delivers stronger SI-SDR across tested arrays, including where the diffusion upscaler fails on a substantially different geometry. The section defines fidelity metrics and compares array-agnostic methods before examining their generalization behavior.

2) Evaluation Metrics:

  • The evaluation uses log-magnitude spectral error, mean spectral coherence, and SI-SDR, which is invariant to global gain mismatches.The metrics assess spectral magnitude fidelity, normalized cross-spectral coherence, and gain-invariant time-domain fidelity, respectively.Source: 2) Evaluation Metrics · p. 8

1) Full- and Semi-Agnostic Methods Evaluation:

  • ADEUPS achieves superior SI-SDR across array configurations without training on them, while outperforming array-agnostic baselines in first-order SI-SDR and nearly all spectral-error cases.It is second-best in coherence for first-order encoding; the authors attribute its advantage over the same-capacity diffusion upscaler to learning a clean prior and applying array-specific operators at inference.Source: 1) Full- and Semi-Agnostic Methods… · p. 8
  • The Diff-Upscaler performs comparably on five-microphone arrays but fails on Project Aria, whose geometry differs from its four- and six-microphone training arrays.The authors connect this contrast to unreliable generalization when supervised training geometries differ substantially from the test geometry.Source: 1) Full- and Semi-Agnostic Methods… · p. 8

2) Array Specific Methods Evaluation:

On a four-microphone array, ADEUPS outperforms array-specific baselines on key SI-SDR comparisons and remains consistent on an unseen configuration. Ablations examine order masking, score conditioning and scheduling, and truncation order.

2) Array Specific Methods Evaluation:

  • ADEUPS exceeds array-specific baselines in SI-SDR for first-order and orders 2–5, and remains consistent on an unseen four-microphone configuration where the baselines fail.For first-order signals, UNet-Encoder leads in coherence and spectral error; at orders 2–5, ADEUPS has comparable coherence and only slightly worse spectral error than Diff-Encoder.Source: 2) Array Specific Methods Evaluation · pp. 8–9, Table III · p. 9

3) Ablation Study:

  • Order masking substantially improves higher-order SI-SDR while maintaining comparable lower-order performance relative to ADEPS.The ablation compares configurations under the same 150-model-evaluation budget.Source: 3) Ablation Study · p. 9, Table IV · p. 10
  • Refinement-score conditioning performs similarly to masking alone, whereas generation-only conditioning degrades low-order performance and reverses the trend across Ambisonics orders.Source: 3) Ablation Study · p. 9
  • A scheduler switches from blended scores to the refinement score below σth, stabilizing sampling after high-order components are established.Figure 2 compares average low- and high-order SI-SDR over iterations with and without the scheduler.Source: 3) Ablation Study · p. 9, Figure 2 · p. 10
  • Performance peaks at Neff = 5, drops noticeably by Neff = 10, and changes minimally from Neff = 10 to Neff = 15.The discrepancy between Neff = 5 and 15 concentrates at high frequencies, where spatial aliasing is pronounced; the Np = 5 prior cannot represent higher-order components, while Neff = 15 is a practical approximation of the infinite sum for speech.Source: 3) Ablation Study · pp. 9–10, Figure 3 · p. 10

V. LISTENING EXPERIMENT

The listening experiment compared ADEUPS with reference and learned systems across three array and acoustic scenarios, using MUSHRA ratings. ADEUPS was not significantly different from ground truth in any screen, while ground truth significantly exceeded the named learned baseline in S1 and S3.

A. Setup

  • The three screens covered single-speaker FOA, dual-speaker FOA, and single-speaker HOA4 across distinct room and microphone-array setups.The scenarios used 4-, 5-, and 6-microphone arrays, respectively.Source: A. Setup · p. 10

B. Methodology

  • Eleven listeners rated stimuli using MUSHRA on a 0–100 scale, with ground truth as reference and linearly encoded FOA as the low anchor.Screen and stimulus order was randomized; processing-method effects were tested with repeated-measures ANOVA.Source: B. Methodology · pp. 10–11, C. Results · p. 11

C. Results

  • Across S1–S3, ADEUPS was not significantly different from ground truth; ground truth significantly exceeded UNet-Encoder in S1 and Diff-Upscaler in S3.In S2, Gen-A scored significantly below both ADEUPS and ground truth.Source: C. Results · p. 11
  • S1 mean scores were ADEUPS 100, ground-truth FOA 88.72, and UNet-Encoder 70.90; ground truth significantly exceeded UNet-Encoder.ADEUPS differed significantly from neither comparator.Source: C. Results · p. 11
  • In Screen 2 (S2), mean scores were 100 for ADEUPS, 96.09 for ground-truth FOA, and 62.68 for UNet-Encoder; Gen-A performed significantly worse than both ADEUPS and ground truth (Bonferroni-corrected p < 0.05).ADEUPS and ground-truth FOA did not differ significantly (p = 0.58).Source: C. Results · p. 11
  • S3 mean scores were ADEUPS 100, ground-truth FOA 91.45, and Diff-Upscaler 81.45; ground truth significantly exceeded Diff-Upscaler.ADEUPS differed significantly from neither ground truth nor Diff-Upscaler.Source: C. Results · p. 11

VI. CONCLUSIONS

  • ADEUPS combines an acquisition-independent prior for ideal HOA signals with array-specific forward modeling, order-masking training, and inference-time score blending.The conclusion reports objective and subjective performance comparable to or exceeding array-agnostic model-based and array-specific data-driven methods, without training on specific array geometries.Source: §VI · p. 11

Related papers

Loading 2610.39185v1…