Source-linked AI summary
The implications of perception as probabilistic inference for correlated neural variability during behavior
Ralf M. Haefner, Pietro Berkes, József Fiser
TL;DR
The paper addresses how cortical feedback and correlated neuronal variability function in perception, where probabilistic-inference models have posed challenges for generating neurophysiologically testable predictions. It uses psychophysical-task structure to derive testable feedback predictions and shows that its model accounts for several difficult 2AFC findings, including divergent choice-probability and psychophysical-kernel time courses.
Problem
Systems neuroscience still faces open questions about the origin and role of correlated neuronal variability and how to generate neurophysiologically testable predictions from probabilistic inference.
Method
The model uses the structure of a psychophysical task within a probabilistic-inference framework to derive testable predictions about feedback and sensory representations.
Results
The model naturally accounts for several findings in the 2AFC task that are difficult for traditional feedforward models, including increasing choice probabilities and decreasing psychophysical kernels linked by a positive feedback loop.
Takeaways & Limitations
Correlated variability can contain task-related information rather than only performance-degrading noise, and sensory and cognitive components can be integrated into physiologically testable decision-making models.
Takeaways & Limitations
The model does not assume optimal brain inference and represents suboptimality through features including sampling-based approximations that converge toward the exact solution only under stated conditions.
Abstract
from arXiv · showhide
This paper addresses two main challenges facing systems neuroscience today: understanding the nature and function of a) cortical feedback between sensory areas and b) correlated variability. Starting from the old idea of perception as probabilistic inference, we show how to use knowledge of the psychophysical task to make easily testable predictions for the impact that feedback signals have on early sensory representations. Applying our framework to the well-studied two-alternative forced choice task paradigm, we can explain multiple empirical findings that have been hard to account for by the traditional feedforward model of sensory processing, including the task-dependence of neural response correlations, and the diverging time courses of choice probabilities and psychophysical kernels. Our model makes a number of new predictions and, importantly, characterizes a component of correlated variability that represents task-related information rather than performance-degrading noise. It also demonstrates a normative way to integrate sensory and cognitive components into physiologically testable mathematical models of perceptual decision-making.
Introduction
The paper uses a controlled 2AFC task to derive testable predictions from probabilistic inference about feedback, sensory responses, and correlated variability. The model reproduces several neural and psychophysical findings while generating predictions that reflect task structure.
- Probabilistic inference has been difficult to translate into neurophysiologically testable predictions because the brain’s general-vision internal model is unknown.
- A controlled task solves this problem by letting experimenters specify the sensory-input generative model and predict how learned structure changes neural responses.The approach treats learning the experimenter-defined model as a perturbation of the unknown general-vision model.
- In 2AFC tasks, the model reproduces task-dependent noise correlations, increasing choice probabilities, and declining psychophysical kernels in some tasks but not others.Choice probability measures correlations between sensory responses and behavior, whereas the psychophysical kernel measures correlations between stimulus and behavior.
- The framework predicts two maxima and two minima in sensory-response noise correlations, with locations set by task-relevant stimuli and amplitudes increasing during perceptual learning.
- It also predicts that evidence weighting during a trial depends on stimulus strength and that these task-structured findings generalize across stimuli, modalities, and sensory areas.
- Because the framework predicts the full statistical structure of neural responses, including higher-level correlations, it guides analysis of high-dimensional population recordings.
Results
The probabilistic-inference framework uses task knowledge and feedback to predict task-dependent sensory correlations and links among noise correlations, choice probabilities, and psychophysical kernels. It accounts for observed patterns while generating predictions about learning, alternative tasks, and temporal evidence weighting.
- Model assumptions: Sensory neurons are modeled as representing beliefs shaped by retinal input and prior task knowledge.The model assumes early visual activity depends on both the retinal image and prior knowledge elsewhere in the brain.
- Correlations between stimulus and behavior: A self-reinforcing sensory–decision feedback loop makes early evidence influence the final choice more strongly than late evidence, producing a decreasing psychophysical kernel.This prediction agrees with a disparity-discrimination result in which CP increased over time while the PK decreased.
- Task-induced correlations: Task-related feedback induces choice correlations and higher noise correlations among neurons supporting the same choice than among neurons supporting opposing choices.Belief fluctuations increase or decrease responses together for neurons supporting the same choice, but in opposite directions for neurons supporting different choices.
- Task-induced correlations: The predicted correlation structure is defined by task-relevant orientations, with peaks and troughs whose amplitude is predicted to increase as task knowledge improves.The model links the correlation amplitude to the degree of task knowledge and predicts stronger top-down influence during perceptual learning.
- Task-induced correlations: For alternative tasks, the model predicts three diagonal peaks and corresponding troughs for 3AFC, but a single peak at the task-relevant stimulus for detection.These predictions extend the framework beyond the two-alternative orientation task.
- Correlations between stimulus and behavior: Choice probabilities increase over time, are largest for neurons tuned to task-relevant orientations, and are predicted to increase with learning.The model also relates CPs to the informativeness of task-aligned neurons and to neurometric thresholds.
Discussion
The paper uses probabilistic inference and task structure to derive testable feedback predictions that explain task-dependent correlations and diverging choice-probability and psychophysical-kernel time courses. The model links sensory and cognitive computations, makes predictions across learning and tasks, and identifies correlated variability as potentially task-related information rather than only noise.
- Framework: Probabilistic inference provides a normative framework for deriving neurophysiologically testable predictions from psychophysical task structure.The framework’s predictions directly reflect the experimenter-defined task, making them easy to test.
- Assumptions and limits: The model’s scope is limited because it is applied to a simulated 2AFC setting and strictly predicts differences between before- and after-learning conditions.The paper identifies extending the model to reaction-time paradigms as future work.
- Empirical findings: In the 2AFC task, the model accounts for empirical findings that are difficult for traditional feedforward processing models to explain.These findings include task-dependent noise correlations and differing time courses of choice probabilities and psychophysical kernels.
- Learning and correlations: The predicted noise-correlation structure has two maxima and two minima located by task-relevant stimuli, and its amplitude increases as perceptual learning improves performance.Across the model parameter regime, task-induced choice probabilities and correlations increase during learning, while the correlation structure retains task-defined locations.
- Assumptions and limits: Finite sampling and mismatch between internal and experimenter-defined task models make the inference process explicitly suboptimal.Finite samples overweight early evidence, while the learned internal model can deviate from the external task model.
- Mechanism: Belief propagation selectively increases responses of neurons supporting the more likely choice, unlike performance-improving attention, and thereby produces choice probabilities.Alternating attention would instead reduce performance and predict an inverse relationship between correlation strength and performance.
- Feedback dynamics: Choice probabilities increase over time while psychophysical kernels decrease because feedback links decision-making neurons with sensory neurons through a positive feedback loop.Choice probabilities also contain a feedforward component, reflected by values above 0.5 at stimulus onset, while the decreasing component follows the psychophysical-kernel time course.
Experimental Procedures
The model represents a two-alternative orientation task with learned task structure, sensory latent variables, and online probabilistic evidence accumulation. Gibbs-sampling simulations generate predictions for sensory responses, correlations, choice probabilities, and psychophysical kernels.
- The task contains exactly two possible decisions, D = 1 and D = 2, with equal prior probability.
- Task learning specifies two task-relevant orientations, with κ ranging from no orientation knowledge to an internal model matching the experimenter’s generative model.κ = 0 denotes no knowledge; κ → 8 denotes perfect learning.
- The model uses circular-Gaussian uncertainty around task-relevant orientations, while κ, λ, and δ determine orientation bandwidth and expected signal saliency.δ = 0 removes task dependence from sensory responses; increasing δ increases expected pattern intensity.
- The sensory representation models V1 neurons as Gabor-shaped feature variables whose noisy linear superposition generates the retinal image.The feature variables can be interpreted as firing rates without a specific spike-production mechanism.
- For dynamic stimuli, the posterior over the decision evolves by accumulating evidence across frames and samples drawn from the posterior over latent orientation variables.The sampling timescale n_s is a free parameter; smaller n_s predicts a faster temporal increase in choice probability.
- The accumulated belief over the correct decision acts as a top-down prior on sensory representations, inducing task-related correlations between sensory neurons.The simulations represent this belief in an abstract decision area rather than with a spike-based implementation.
Supplementary Information
The supplementary model defines a sparse-coding image representation and derives conditional distributions for Gibbs sampling. Neural responses represent posterior uncertainty and inherit correlations from correlated latent posteriors.
- Images are represented as column vectors, and rescaled projective fields are combined into a matrix G for the generative model.The projective fields are Gabor functions differing only in orientation and are normalized to unit 2-norm.
- The generative model includes a categorical decision variable D taking values 1 or 2.
- Gibbs sampling uses conditional distributions for D, each orientation variable g_k, and each sensory variable x_k.These conditionals combine the relevant likelihood and prior factors in the hierarchical model.
- The image likelihood depends on the reconstruction residual y − Gx, while the sensory variables follow cut-Gaussian distributions with analytically specified means and variances.
- In neural sampling, response variability represents uncertainty about a scalar variable, and correlated posterior variables imply correlated neural responses.
S1.2 Correlations between variables in the internal model imply correlated neural responses
The model links dependencies in the internal posterior to correlated neural variability and predicts how model mismatch and task-related beliefs shape performance and response correlations. It also provides a rational process account connecting sequential sampling, sensory variance, choice conditioning, and task-dependent correlations.
- Dependencies between variables in the joint posterior produce correlated variability in their samples and, consequently, correlated neural responses.Figure S1 identifies posterior dependencies as the source of correlated variability.
- For model-match parameter a=1, inference is correct, performance is maximal, and posterior samples of g1 and g2 are perfectly anticorrelated.When the internal and external models are identical, the posterior concentrates on opposing binary states.
- For maximal mismatch, a=0.5, the posterior becomes input-independent for ambiguous stimuli, performance reaches chance, and samples of g1 and g2 are uncorrelated.Intermediate values 1/2 < a < 1 interpolate between the performance and correlation extremes.
- In a 2AFC task, sensory response variance is smaller when conditioned on the behavioral decision, while unconditioned variability is higher because choice-conditioned means differ.The framework attributes these variance patterns to variability linked to internal beliefs and decisions.
- The rational process model combines the task’s generative model with sequential neural sampling to explain task-dependent correlations and the sub-optimality of decreasing psychophysical kernels.The sequential sampling mechanism provides a process-level account of these behavioral and neural effects.
- Probabilistic inference predicts enhanced responses to stimuli compatible with internal decisions or high priors, contrasting with standard predictive-coding predictions of diminished correctly predicted responses.The paper connects this enhancement to empirical choice probabilities and task-dependent correlation measurements.