Source-linked AI summary
Bidirectional representational alignment between biological and artificial neural networks
Samuel Kostousov, Abhinn Kaushik, Brokoslaw Laschowski
TL;DR
Representational alignment between biological and artificial neural networks is asymmetric, raising whether representational geometry contributes to this gap. The paper steers spectral geometry during learning and finds reduced asymmetry with improved bidirectional predictivity.
Problem
Representational alignment is asymmetric, motivating investigation of whether representational geometry contributes to bidirectional alignment.
Method
The framework combines spectral regularization during contrastive-model training with bidirectional predictivity and representational-geometry analyses.
Results
Steering spectral geometry increased reverse predictivity with modest forward-predictivity reductions, reorganized the shared subspace, and reduced effective dimensionality.
Takeaways & Limitations
Forward–reverse asymmetry can be systematically reduced by changing learned representational organization, supporting geometry as a steerable factor in bidirectional alignment.
Takeaways & Limitations
Evaluation used one self-supervised contrastive ResNet-50 layer and one neural benchmark, so generalization across architectures, objectives, layers, and datasets remains untested.
Abstract
from arXiv · showhide
Recent work has shown that representational alignment between biological and artificial neural networks is asymmetric: model representations predict neural responses much better than neural responses predict model representations. This asymmetry raises the question of whether representational geometry contributes to bidirectional representational alignment. We hypothesized that steering representational geometry during training can systematically influence bidirectional alignment. To test this hypothesis, we developed a computational framework that integrates spectral regularization with bidirectional predictivity analyses. As an initial demonstration, we evaluated our framework using self-supervised contrastive vision models. Steering the spectral geometry of the learned representations substantially increased reverse predictivity with modest reductions in forward predictivity, yielding a 55% relative improvement in bidirectional predictivity. These improvements were accompanied by reduced effective dimensionality and a reorganization of the shared representational subspace, within which forward and reverse predictivity became approximately symmetric at intermediate spectral exponents. Overall, these findings demonstrate that representational geometry can be systematically steered to modulate bidirectional representational alignment between biological and artificial neural networks.
I. INTRODUCTION
The introduction frames representational alignment as a way to study computational principles across biological and artificial neural networks. It identifies forward–reverse predictivity asymmetry and motivates spectral steering as a means to influence bidirectional alignment.
- Motivation: Representational alignment compares internal neural-network representations to study computational principles underlying intelligence beyond external behavior.The approach is presented as relevant to both machine learning and computational neuroscience.
- Motivation: Task-performance improvements do not always produce comparable gains in representational alignment, suggesting that learned-representation properties also influence alignment.
- Bidirectional predictivity: Forward predictivity measures model-to-neural prediction, whereas reverse predictivity measures neural-to-model prediction; these metrics reveal significant asymmetry favoring forward prediction.
- Approach: The study hypothesizes that steering representational geometry during learning can systematically influence bidirectional alignment and tests this with spectral steering and predictivity analyses in self-supervised contrastive vision models.The framework evaluates alignment with representations in visual cortex.
II. METHODS
The framework combines spectral regularization with bidirectional predictivity analyses to study how representational geometry relates to alignment between biological and artificial neural networks. It was demonstrated with self-supervised contrastive vision models, where forward–reverse asymmetry is pronounced.
- Model implementation: The initial demonstration implemented the framework using self-supervised contrastive vision models.Contrastive learning was selected because prior work found particularly pronounced forward–reverse predictivity asymmetry in this model class.
- Framework: The framework combines spectral regularization, which steers representational geometry, with forward and reverse predictivity analyses.During training, spectral decay exponent regularization is added to a standard learning objective; during validation, bidirectional predictivity is evaluated on stimuli.
- Study design: The method evaluates whether steering spectral geometry during training changes bidirectional representational alignment.The computational framework was designed to study the relationship between learned representational geometry and alignment between biological and artificial neural networks.
A. Dataset
The framework was trained self-supervised on a balanced ImageNet-1k subset and evaluated against a macaque inferotemporal cortex benchmark.
- A. Dataset: The study used 100 images from each of ImageNet-1k’s 1,000 classes, with balanced 80/20 training and validation splits, and evaluated models on macaque inferotemporal cortex.Class labels were not used during self-supervised contrastive training; the objective relied on correspondences between independently augmented views.
B. Model
The framework was implemented with SimCLR using a ResNet-50 encoder and a two-layer projection head, while spectral regularization was introduced by modifying only the training objective. Regularization targeted a specific batch-normalization layer whose activations were used for subsequent analyses.
- Model architecture: SimCLR used a ResNet-50 encoder and a two-layer projection head mapping image representations into a 128-dimensional contrastive embedding space.The baseline was trained with the standard SimCLR objective.
- Training objective: The regularized condition kept the architecture, optimization procedure, and evaluation protocol fixed while modifying only the training objective through spectral regularization.This isolated the effect of steering representational geometry through the objective.
- Analysis layer: Spectral regularization targeted encoder.layer4.0.bn1, and its activations were flattened into an activation matrix A ∈ R^B×D for subsequent analyses.Here, D = C × H × W denotes the flattened feature dimension, and the layer is the first batch-normalization layer in the encoder’s final residual stage.
C. Training
The framework trained self-supervised contrastive vision models with the standard SimCLR protocol while adding spectral regularization to steer representational geometry. Spectral decay was controlled through a target exponent estimated from activation spectra and introduced after an initial training period.
- Training protocol: Models followed the standard SimCLR protocol for 200 epochs using two independently augmented views per image.Training used stochastic gradient descent with momentum, cosine learning-rate decay, and a 10-epoch warmup.
- Contrastive objective: The InfoNCE objective maximized agreement between augmented views of the same image while minimizing agreement between views of different images.In spectral regularization experiments, the temperature hyperparameter was fixed at τ = 0.2.
- Spectral regularization: Spectral regularization used encoder activations to penalize deviations from a target spectral exponent, steering representational geometry during learning.The exponent α was estimated by ordinary least squares on the log–log spectrum over principal components 5–15.
- Spectral targets: The target spectral exponent ˆα was fixed within each run, initialized to approximately 1, and supplemented with additional target values.Regularization began after the first 10 training epochs to improve training stability.
D. Inference
The inference analysis quantified bidirectional representational alignment through forward and reverse predictivity from model activations and neural responses. It also examined common versus unique units and effective dimensionality to characterize underlying mechanisms.
- Inference: Forward predictivity was estimated by fitting ridge regressions from selected-layer model activations to each recorded neuron using fivefold cross-validation.Activations were extracted for each stimulus with neural data.
- Inference: Reverse predictivity was estimated by fitting ridge regressions from neural responses to each model unit.
- Inference: Predictivity was quantified as squared reliability-corrected Pearson correlation on held-out data, with correction based on the geometric mean of split-half reliabilities.The corrected correlation was converted to a percentage.
- Inference: The analysis compared the top and bottom 20% of model units by reverse predictivity, termed common and unique units, respectively.Forward and reverse predictivity were recalculated for each subset.
E. Experiments
Experiments varied target spectral exponents and regularization strengths while holding other training hyperparameters fixed. Analyses were conducted after training and organized by measured rather than target spectral exponent because target values were not always achieved.
- The framework was evaluated across target spectral exponents ˆα and spectral regularization strengths λ, with all remaining training hyperparameters held fixed.
- Evaluations were performed after training because of the high computational cost of the analyses.
- Analyses were organized by measured spectral exponent α rather than target exponent ˆα because lower regularization strengths did not always reach the intended target.
III. RESULTS
Spectral regularization systematically steered learned representational geometry and increased reverse and bidirectional predictivity despite modest reductions in forward predictivity. At intermediate spectral exponents, forward–reverse asymmetry was nearly eliminated within the shared model–neural subspace, alongside reduced effective dimensionality.
- Spectral geometry: Measured spectral exponents increased with target spectral exponents, demonstrating that spectral regularization systematically steered learned representational geometry.Stronger regularization produced closer agreement between measured and target values, with no systematic differences across evaluated targets.
- Reverse predictivity: 29% reverse explained variance in the highest-α bin, up from 12% at the unregularized baseline.Reverse predictivity generally increased with measured spectral exponent and regularization strength, including across the population of model units.
- Forward predictivity: Forward explained variance generally decreased under spectral regularization, while varying comparatively less across measured α at fixed regularization strength.Forward-predictivity distributions shifted toward lower explained variance relative to the baseline.
- Bidirectional predictivity: 31% bidirectional predictivity, up from approximately 20%, corresponded to a relative improvement of 55%.Bidirectional predictivity increased with measured spectral exponent as higher reverse predictivity reduced asymmetry despite lower forward predictivity.
- Shared representational subspace: 45% reverse predictivity in common units coincided with 37% forward predictivity in the highest α bin, while intermediate α nearly eliminated forward–reverse asymmetry.Common units were the top 20% ranked by reverse predictivity; unique units showed substantially smaller reverse-predictivity gains, and effective dimensionality decreased as measured α increased.
IV. DISCUSSION
The study shows that steering spectral geometry can reduce forward–reverse asymmetry and improve bidirectional alignment, alongside reorganized shared representations and reduced effective dimensionality. The discussion frames spectral exponent as one associated property rather than a sole determinant and identifies broader geometric analyses as future work.
- Main contribution: Steering spectral geometry increased reverse predictivity, modestly reduced forward predictivity, and improved overall bidirectional predictivity.These changes reduced asymmetry between the two directions.
- Mechanisms: The alignment improvements accompanied reorganization of the shared representational subspace and reduced effective dimensionality.These changes provide insight into potential mechanisms underlying improved representational alignment.
- Asymmetry: Within the common representational subspace, forward and reverse predictivity became approximately symmetric at intermediate spectral exponents.The findings indicate that forward–reverse asymmetry can be systematically steered by changing learned-representation organization.
- Spectral exponent: Bidirectional alignment continued improving beyond the previously reported visual-cortex spectral exponent of approximately α = 1.Thus, matching α = 1 alone was not sufficient to maximize alignment; increasing spectral exponents were also associated with reduced effective dimensionality and improved reverse predictivity within the common subspace.
- Limitations: Similar spectral exponents can accompany different geometric or statistical properties, and forward predictivity varied with regularization strength among models with similar measured exponents.Therefore, the measured spectral exponent is one property associated with alignment changes, not their sole causal determinant.
- Future directions: Future work should study geometric properties beyond the spectral exponent and develop more selective steering methods, including adaptive or scheduled spectral regularization.These approaches could help disentangle geometry effects from those introduced by regularization strength.