Source-linked AI summary
Framing Spatial Hearing within the Free Energy Principle: A Hierarchical Model of Perceptual Learning
Roberto Barumerli, Nils Meyer-Kahlen, Lorenzo Picinali
TL;DR
The model formulates sound localisation as estimating an invisible source’s direction, separating lateral and polar auditory cues and beliefs about their precision and hemifield. A hierarchical listener model uses visual feedback to drive gradient descent on variational free energy, updating lateral and polar precision and hemifield disambiguation across trials. Lateral precision and hemifield disambiguation increase, while polar precision also rises, especially for listeners with the lowest initial precision.
Problem
The model formulates sound localisation as estimating an invisible source’s direction, separating lateral and polar auditory cues and beliefs about their precision and hemifield.
Source: p. 2Method
A hierarchical listener model uses visual feedback to drive gradient descent on variational free energy, updating lateral and polar precision and hemifield disambiguation across trials.
Source: p. 2Results
Lateral precision and hemifield disambiguation increase, while polar precision also rises, especially for listeners with the lowest initial precision.
Source: p. 2, §A.3 · p. 8Takeaways & Limitations
The work establishes spatial hearing as a tractable domain for the free energy principle and motivates future experiments with full action-perception cycles.
Source: p. 2The study has a small cohort and models a perception-only loop, with pointing as read-out and external feedback.
Source: p. 2
Abstract
from arXiv · showhide
Spatial hearing relies on dynamic internal representations to decode spatial information from ever-changing acoustic conditions. A fundamental example is sound localisation, where listeners infer the direction of an invisible sound. Bayesian inference provides a computational account of such a localisation process; however, modelling efforts are largely limited to static listening scenarios. Instead, the free energy principle (FEP) describes perception, learning, and action as joint minimisation of variational free energy (VFE), providing a principled framework to account for dynamic conditions. Here, we take a first step toward applying the FEP to spatial hearing. We characterise how listeners' beliefs evolve across trials, in which they estimated sound direction and received subsequent visual feedback on each trial. Listeners are described by a hierarchical generative model where visual feedback drives gradient descent on VFE, updating beliefs over three independent quantities: lateral and polar localisation precisions and hemifield disambiguation. The model captures individual learning trajectories and reveals components with different dynamics: lateral precision improves, as does hemifield disambiguation, consistent with increased sensitivity of the auditory pathway in decoding vertical direction. Notably, polar precision also increases, most for listeners with the lowest initial precision, suggesting that the observed vertical improvement is driven by both hemifield disambiguation and polar precision. This work establishes spatial hearing as a tractable domain for FEP and motivates future closed-loop experiments with full action-perception cycles.
Generative model
The listener model factorises spatial responses into lateral and polar components, with a two-component polar mixture representing hemifield ambiguity.
Generative model
- Directions use interaural-polar coordinates, separating lateral angle α from polar angle β.The lateral angle defines the cone of confusion, within which the polar angle is decoded from spectral cues.Source: Generative model · p. 6
- The listener model factorises into lateral and polar response components.Source: Generative model · p. 6
- Polar responses are a mixture over correct and mirror hemifields, with shared concentration scaled by cos2(αr) and weight w for the correct hemifield.The scaling accounts for geometric compression of the cone of confusion at lateral angles.Source: Generative model · p. 6
Free energy and natural gradient
Belief revision minimizes variational free energy, which under the stated approximation reduces to negative log likelihood; natural-gradient steps update the beliefs.
Free energy and natural gradient
- At trial i, variational free energy is defined for a recognition density over beliefs θ = (κL, κP, w).Source: Free energy and natural gradient · p. 6
- With a Gaussian Laplace approximation and flat prior, the complexity term is constant, so free energy reduces to negative log likelihood plus a constant.Source: Free energy and natural gradient · p. 6
- Natural-gradient descent scales each belief update by its learning rate and inverse Fisher information, and the factorised likelihood decouples the three updates.The hemifield weight is updated through logit ψ, then recovered as w = σ(ψ).Source: Free energy and natural gradient · p. 6
Update equations
The update equations use response likelihoods for the polar mixture and responsibility-weighted alignment, while visual feedback supplies the true cone angle for learning.
Update equations
- The polar likelihood combines correct- and mirror-hemifield von Mises components using weight wi and scaled concentration κP,i cos2(αt).Source: Update equations · p. 7
- ri is the responsibility-weighted cosine alignment between the observed response and target direction.Source: Update equations · p. 7
- Perception conditions on inferred cone αr, but visual feedback reveals the true direction, so the κP update uses cos2(αt).The logit parameterisation keeps w between zero and one without projection.Source: Update equations · p. 7
A.2 Parameter estimation
Initial beliefs are estimated from baseline responses, while subject-specific learning rates are fitted to sequential responses in the feedback block.
A.2 Parameter estimation
- Initial beliefs θ0 are estimated per subject by maximum likelihood from the last 100 baseline trials without feedback.Source: §A.2 · p. 7
- The three learning rates are the model’s only free parameters and are fitted per subject by minimizing mean negative log likelihood in the feedback block.Fitting uses Bayesian adaptive direct search and sequential updates.Source: §A.2 · p. 7
- Because the likelihood factorises, ηL is estimated independently from ηP and ηw.Source: §A.2 · p. 7
A.3 Model comparison and robustness
Held-out comparisons support learning both hemifield disambiguation and polar precision, while robustness tests show that polar precision still rises without cos² scaling. The scaling affects the size of polar learning and preserves the model’s geometric treatment of lateral extremes.
Robustness to the cos² scaling
- Without cos² scaling, median ηP fell from 0.015 to 0.006; κP still rose by 0.6 to 2.8, while summed in-sample ΔLL was −53 nats.The rise in κP therefore did not depend on geometric weighting, although the unscaled model fit worse overall.Source: §A.3 · p. 8, Table 1 · p. 8
- The unscaled model violates the problem geometry because the cone of confusion collapses at α = ±90°, where polar angle is undefined.At more lateral positions, polar errors also become less informative about spectral decoding.Source: §A.3 · p. 8
Model comparison
- 167 nats (freezing κP) and 82 nats (freezing w) were held-out log-likelihood losses; hemifield learning was required in every subject, polar learning in all but NH17.NH17’s polar-precision learning rate was indistinguishable from zero.Source: §A.3 · pp. 7–8, Table 1 · p. 8