PaperlayerView exact revision on arXiv

Source-linked AI summary

HRTF Upsampling and Personalization via Neural Operators

Meenakshi Krishnan, You Zhang, Ramani Duraiswami, C. Phillip Brown

arXiv:2610.20400v1cs.SDcs.LG

TL;DR

Sparse HRTF measurements make dense personalized fields costly to acquire, while rendering may require arbitrary directions. SPARO maps sparse observations through shared spherical latent modes to queryable HRTFs, delivering strong results on SONICOM and HUTUBS.

  • Problem

    Dense individualized HRTFs require costly measurements at hundreds of directions, and rendering may require directions outside the measured grid.

    Source: §1 · p. 1

  • Method

    SPARO learns a shared sparse-to-dense operator that maps observations through spherical latent modes to HRTFs at arbitrary query directions.

    Source: §1 · p. 1, §2.3 · pp. 2–3

  • Results

    Across SONICOM and HUTUBS, SPARO leads SONICOM LSD and ILD at every sparsity level, with 18% lower LSD and 43% lower ILD error at M=100, and lower LSD at every HUTUBS sampling level.

    Source: §3.2 · p. 4, §3.3 · p. 4, §4 · p. 4

  • Takeaways & Limitations

    HUTUBS results further support operator learning as an effective approach for personalized HRTF reconstruction.

    Source: §4 · p. 4, §3.3 · p. 4

    Right-ear mirroring assumes approximate bilateral symmetry after reflection about the median plane.

    Source: §2.4 · p. 3

Abstract

from arXiv · show

Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formulate HRTF upsampling as an operator learning problem and introduce SPARO, a neural operator that maps sparse measurements at a set of spatial directions to the HRTF field, a continuous representation that can be queried at arbitrary positions. SPARO projects the measurements onto a learned set of spherical latent modes, processes the latent modes with Transformer blocks, and decodes the resulting representations at query directions. On SONICOM, using the LAP24 measurement configuration, SPARO attains the best log-spectral distortion (LSD) and interaural level difference (ILD) error at every sparsity level from 3 to 100 measurements (18% lower LSD and 43% lower ILD error than the strongest baseline at 100 measurements) while remaining competitive in interaural time difference (ITD) error. It reconstructs a dense HRTF 15 to 373 times faster than the leading neural-field method RANF. We further validate the approach on HUTUBS, demonstrating its effectiveness across HRTF datasets.

1. INTRODUCTION

Sparse HRTF measurements make dense personalized fields costly to acquire, while rendering may require directions outside the measurement grid. SPARO learns a shared operator that maps sparse observations to queryable HRTF fields, and achieves the lowest LSD and ILD errors across SONICOM sparsity levels.

1. INTRODUCTION

  • Personalized HRTFs matter because anatomy-specific filtering affects localization, externalization, and immersion, with spatial rendering also relevant to hearing assistance.Source: §1 · p. 1
  • Dense HRTF acquisition requires hundreds of directions and is costly, while practical rendering may need arbitrary directions beyond the measurement grid.Source: §1 · p. 1
  • Interpolation and spherical harmonics degrade under high sparsity, while neural fields often adapt listener-specific representations from sparse measurements.Source: §1 · p. 1
  • SPARO maps sparse observations through learned spherical modes and latent transformations, then interpolates the representation at arbitrary query directions.Source: §1 · p. 1, Figure 1 · p. 1, Figure 2 · p. 2
  • On SONICOM under LAP24, SPARO achieves the lowest LSD and ILD errors at every sparsity level and remains competitive in ITD error.Source: §1 · p. 2

2. METHODS

The methods frame HRTF upsampling as mapping sparse measurements to arbitrary query directions, using learned spherical modes to represent and decode the field. Separate magnitude and ITD models are personalized at test time, with right-ear measurements mirrored under an approximate-symmetry assumption.

2.1. Preliminary and Notations: HRTF Upsampling

  • An HRTF is a complex-valued transfer function of frequency and source direction, encoding magnitude and phase.Its time-domain counterpart is the head-related impulse response (HRIR).Source: §2.1 · p. 2
  • The upsampling task estimates a test subject’s HRTF over a dense direction grid from measurements available only at sparse directions.Methods model log-magnitude and ITD separately; predicted magnitudes yield minimum-phase HRIRs, with predicted ITD applied as a fractional delay.Source: §2.1 · p. 2

2.2. Formulating HRTF Upsampling as Operator Learning

  • SPARO learns an operator mapping sparse observations of a directional field on the sphere to field values at arbitrary query directions.The formulation is applied to both log-magnitude and scalar ITD fields.Source: §2.2 · p. 2

2.3. SPARO Architecture

  • A branch MLP embeds observed direction-value pairs, and a trunk MLP embeds query directions; learned spherical modes aggregate observations before Transformer processing.Each mode has a learnable spherical center and temperature; Transformer processing operates over the latent modes, independent of the number of output queries.Source: §2.3 · p. 2
  • The decoder mixes transformed latent modes according to angular proximity to shared learned spherical centers, producing representations at arbitrary query directions.A direct query-to-observation cross-attention pathway lets each prediction also attend to the original sparse measurements.Source: §2.3 · p. 3

2.4. Implementation and Test-time Personalization

  • SPARO predicts HRTF log-magnitude and ITD separately from sparse measurements, then combines the predictions to reconstruct the HRTF.Right-ear azimuths are reflected to share a left-ear convention, assuming approximate bilateral symmetry after reflection.Source: §2.4 · p. 3
  • Test-time optimization updates the initial latent representation and output MLP using observed measurements, while keeping the Transformer blocks and remaining parameters fixed.The objective combines observed-direction relative L2 error with penalties that keep adapted parameters near pretrained values.Source: §2.4 · p. 3

3. EXPERIMENTS AND RESULTS

Across SONICOM and simulated HUTUBS, SPARO performs well for sparse HRTF upsampling, with strong LSD and ILD results on SONICOM and lower LSD than baselines across HUTUBS sampling levels. Ablations and inference-cost comparisons examine its components and efficiency.

3.1. Experimental Setup

  • Evaluation uses the three objective metrics defined in the LAP24 challenge: LSD measures spectral fidelity between upsampled and reference HRTFs, while ILD and ITD errors are mean absolute errors in dB and µs, respectively.Upsampling is evaluated at four sparsity levels, M ∈ {3, 5, 19, 100}, using LAP24 measurement configurations. A single model trained jointly across all sparsity configurations is evaluated at every level without retraining.Source: §3.1 · p. 3, §3.2 · p. 4

3.2. Experiments on the Measured SONICOM Dataset

  • 18% lower LSD (2.3 dB against 2.8 dB at M=100) and a best-case 43% lower ILD error at M=100; ITD error is competitive across all sparsity levels.SPARO achieves the lowest LSD at all four SONICOM sparsity levels, with 7–14% gains at M=3, 5, and 19; Fig. 3 shows predicted and reference magnitude spectra, including at M=3.Source: §3.2 · p. 4, Figure 3 · p. 4

3.3. Experiments on the Simulated HUTUBS Dataset

  • 3.39 dB LSD from six observed directions; SPARO has lower LSD than SUpDEq and RANF for all ten held-out subjects at every sampling level.The six-direction result corresponds to roughly 4–6× fewer measurements than the baseline needs to reach that LSD level; SPARO’s advantage grows in sparser regimes.Source: §3.3 · p. 4, Table 2 · p. 4

3.4. Ablations

  • TTO improves LSD at every HUTUBS sampling level, while LCA’s benefit grows with M; without TTO, SPARO still outperforms baselines at all but the densest HUTUBS level.The ablation reports degradation when either component is removed across most sampling levels.Source: §3.4 · p. 4, Table 3 · p. 4

3.5. Computational Efficiency

  • 15–373× speedups over RANF; SPARO’s inference cost is independent of M, at 0.56–0.59 s with TTO or 12 ms without, while RANF rises from 8.7 s at M=3 to 207.2 s at M=100.SPARO’s TTO updates only the latent representation and output head for 20 steps; RANF adapts parameters over 500 epochs with backpropagation through the full network and retrieved subjects.Source: §3.5 · p. 4, Table 4 · p. 4

4. CONCLUSION

SPARO frames HRTF spatial upsampling as operator learning, mapping sparse measurements to arbitrary query directions through learned spherical modes. It achieves the lowest LSD and ILD error among compared methods at all four SONICOM sparsity levels, with competitive ITD and substantially lower test-time cost than neural-field methods.

  • SPARO maps sparse HRTF measurements to arbitrary query directions through a learned spherical mode representation.Source: §4 · p. 4
  • SPARO achieves the lowest LSD and ILD error among compared methods at all four SONICOM sparsity levels, with competitive ITD.Source: §4 · p. 4
  • SPARO has substantially lower test-time cost than neural-field methods, and HUTUBS results further support operator learning for personalized HRTF reconstruction.Source: §4 · p. 4

Related papers

Loading 2610.20400v1…