Source-linked AI summary
Graph Neural Network for Wide-Area 6DoF Ambisonic Rendering
Gioele Greco, Filippo Marri, Mirco Pezzoli, Fabio Antonacci
TL;DR
Sparse, irregular Ambisonic arrays make translational 6DoF rendering difficult, and geometric DirAC interpolation cannot adapt to the observed scene. The paper predicts scene-dependent interpolation weights with a graph model while retaining analytical DirAC reconstruction; evaluations report improved energy reconstruction and radiation-pattern similarity across unseen conditions.
Problem
Sparse, irregular Ambisonic arrays make translational rendering difficult, while existing DirAC interpolation uses geometric weights that do not depend on the observed acoustic scene.
Source: §1 · p. 1Method
A GNN predicts positive, query-conditioned weights from array features and relative geometry, then an analytical DirAC layer reconstructs target parameters.
Source: §3 · p. 2Results
Takeaways & Limitations
The method keeps rendering within an interpretable parametric DirAC pipeline while allowing interpolation to depend on the measured scene.
Source: §6 · pp. 4–5Evaluation beyond simulated RIRs and perceptual tests are still needed to assess real-measurement performance and whether the reported improvements are audible.
Source: §6 · p. 5
Abstract
from arXiv · showhide
Wide-area 6DoF rendering from sparse Ambisonic recordings is commonly addressed by interpolating parametric spatial descriptors rather than the waveforms themselves. This preserves the fidelity of a nearby measured signal and avoids phase artifacts caused by mixing distant arrays. However, existing DirAC-based approaches rely on geometric interpolation weights that do not depend on the acoustic scene observed by the microphones. We propose a lightweight Graph Neural Network (GNN) that refines only this interpolation stage. The known Ambisonic arrays and the query position are represented as a graph whose nodes contain local DirAC features and whose edges encode relative distance and directional relations. The network predicts positive query-conditioned weights over the measured arrays, while the target energy, diffuseness, and direction of arrival are reconstructed by an analytical DirAC layer. Experiments on unseen rooms, unseen speech signals, and unseen array configurations show that the proposed weighting improves energy reconstruction and radiation-pattern similarity with respect to baseline interpolation.
1. INTRODUCTION
Wide-area 6DoF rendering is challenging when sparse, irregular Ambisonic arrays leave the sound field unknown at the listener’s position. The paper refines DirAC interpolation with a scene-dependent GNN while retaining analytical reconstruction.
1. INTRODUCTION
- Sparse, irregular Ambisonic arrays make translational rendering difficult because the sound field must be estimated where no microphone is available.Source: §1 · p. 1
- DirAC interpolation avoids mixing distant waveforms but uses geometric weights that cannot adapt to the acoustic scene observed by the arrays.Source: §1 · p. 1
- The proposed GNN refines only interpolation, using measured DirAC descriptors and relative sensor geometry to predict scene-dependent weights.Source: §1 · p. 1
- Tests on unseen rooms, speech signals, and array configurations show improved energy reconstruction and FOA radiation-pattern similarity.Source: §1 · p. 1
2. PROBLEM FORMULATION
The formulation starts from known Ambisonic arrays and a query position, then interpolates DirAC parameters to support rendering there. It preserves the nearby-recording rendering strategy while motivating scene-dependent weights in place of geometric ones.
2. PROBLEM FORMULATION
- The problem is to render at query position q from M known Ambisonic arrays at positions S, each recording a higher-order Ambisonics signal.Source: §2 · p. 2
- The existing method interpolates DirAC parameters at the query using geometric weights from the enclosing simplex.Source: §2 · p. 2
- The resulting parameters adapt a nearby measured recording for binaural rendering, avoiding comb-filtering from direct mixing; the remaining task is scene-dependent weighting.Source: §2 · p. 2
3. GRAPH MODEL
The graph model connects the known arrays and query position, using relative geometry and local acoustic information to produce interpolation weights. Its design avoids dependence on absolute coordinates or sensor ordering.
3. GRAPH MODEL
- A complete graph connects the M measured-array nodes and the query node so every node can exchange information with every other node.Source: §3 · p. 2
- The graph is evaluated independently for each time-frequency bin, and a normalized frequency tag makes predicted weights frequency-conditioned without cross-bin exchange.Source: §3 · p. 2
- Absolute coordinates are excluded; relative distances and directional relations encode sensor spacing and local flow alignment in a coordinate-free representation.Source: §3 · p. 2
- A shared head predicts positive, unnormalized weights over measured arrays; an analytical reconstruction layer converts them to DirAC parameters.Source: §3 · p. 2
- The network learns how to combine measured nodes, while the acoustic descriptors remain generated by closed-form equations.Source: §3 · p. 2
4. TRAINING AND TEST PROCEDURE
The procedure trains the GNN on randomized FOA room and speech examples, evaluates it on held-out rooms, speech, and array layouts, and compares reconstruction metrics against Delaunay interpolation.
4.1. Network configuration.
- The reported network uses two message-passing layers, 48-dimensional node representations, and 27,121 trainable parameters, with fully connected message passing scaling as O(M^2).A shared node-wise MLP produces one log-weight per measured node before the softplus transformation.Source: §4.1 · p. 3
4.2. Training
- Training randomizes rooms, speech excerpts, and receiver-grid sampling, while supervision is applied to reconstructed query descriptors rather than directly to the predicted weights.Training uses simulated FOA room impulse responses from HARP and dry speech from TIMIT.Source: §4 · p. 2, §4.2 · p. 3
- The training loss combines energy, direct-energy, and spatial-regularization terms, with query-position perturbations penalized to encourage smooth predicted weights.Microphone and query positions are also jittered by 0.1 m during training.Source: §4.2 · p. 3
4.3. Test
- Across 25 unseen rooms, three five-array layouts per room yield 75 test configurations, using the same configurations for the GNN and baseline.Unseen TIMIT excerpts are convolved with the corresponding FOA room impulse responses.Source: §4.3 · p. 3
4.4. Evaluation Metrics
- Evaluation measures normalized reconstruction error for energy and diffuseness, and weighted structural correlation of the rendered directional-energy pattern.The pattern metric compares predicted and ground-truth directional-energy maps and accounts for polar grid distortion.Source: §4.4 · p. 3
- The Delaunay baseline uses the query's enclosing simplex and barycentric weights; Fig. 1 maps GNN-minus-baseline differences, with positive values indicating improvement.Its panels report energy NMSE, diffuseness NMSE, and radiation-pattern similarity differences.Source: §4.4 · p. 3, Figure 1 · p. 4
5. RESULTS
The GNN improves energy reconstruction and radiation-pattern similarity over baseline interpolation, while diffuseness stays close to the Delaunay baseline. A qualitative example shows the GNN better retaining the reference pattern’s dominant lobe and directional shape.
5. RESULTS
- Energy NMSE gains reach approximately ∆NMSEE ≃3dB locally, while baseline improvements over the GNN reach at most ≃0.1dB.The GNN improves energy NMSE over a large portion of the evaluated region, though gains are not uniform.Source: §5 · p. 4
- Diffuseness closely matches the Delaunay baseline, with median differences on the order of 10−3 and local variations of about ±5 × 10−3.The baseline is already strong for diffuseness, and the GNN reproduces its behavior across unseen rooms, speech signals, and array configurations.Source: §5 · p. 4
- Radiation-pattern similarity gains reach ∆P ≃0.1 locally, with median improvement positive over almost the entire domain.Gains are not uniform across rooms and array selections; the GNN better estimates angular structure used by the DirAC rendering stage across unseen rooms, speech signals, and array configurations.Source: §5 · p. 4
- In the Fig. 2 example, the GNN better follows the ground-truth dominant lobe and overall directional shape than the smoother, more symmetric baseline.The caption compares ground truth, baseline, and GNN at one query position and frequency band; color encodes normalized energy in dB.Source: §5 · p. 4, Figure 2 · p. 4
- These findings support replacing fixed geometric weights with scene-dependent graph weights while preserving analytical DirAC reconstruction.The reported gains are in energy and directional consistency, while diffuseness remains close to the baseline.Source: §5 · p. 4
6. CONCLUSION
The paper refines wide-area DirAC interpolation with a graph model while keeping target-parameter reconstruction analytical. Results show gains in energy and directional consistency, and the small model is compatible with fast inference.
6. CONCLUSION
- The graph-based refinement replaces fixed geometric interpolation weights with scene-dependent weights while retaining the parametric rendering pipeline.The sensor network is represented explicitly, and the feature design is coordinate-free.Source: §6 · p. 4
- Positive node weights feed analytical reconstruction of target DirAC parameters, keeping the output interpretable between classical interpolation and data-driven sound-field reconstruction.The reported gains are clear for energy reconstruction and directional consistency, while diffuseness remains close to the Delaunay baseline.Source: §6 · p. 5
- The small model is compatible with fast inference and future scene-specific adaptation as new audio becomes available.The proposed extensions include scene-adaptive fine-tuning on a short calibration recording and cross-frequency modeling.Source: §6 · p. 5
- Evaluation remains limited to simulated RIRs; future work will test real multi-point Ambisonic measurements and use listening tests to assess audible improvements.Further proposed directions include coupling neighboring frequency bins and fine-tuning on a target-scene calibration recording.Source: §6 · p. 5