PaperlayerView exact revision on arXiv

Source-linked AI summary

Human alignment of neural network representations

Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen, Simon Kornblith

arXiv:2211.01201v5cs.CVcs.AIcs.LGq-bio.NC

TL;DR

The paper investigates whether neural-network representations align with human mental representations inferred from behavioral similarity judgments. It evaluates models across three datasets and two tasks, finding that training data and objective matter more than model scale or architecture. The results suggest scaling alone may not yield conceptual representations matching those used by humans.

  • Problem

    Although computer-vision models perform strongly on vision tasks, how closely their learned conceptual representations align with human mental representations remains unclear.

    Source: §1 · p. 1

  • Method

    The study compares neural-network representations with human similarity judgments across triplet and multi-arrangement tasks, including linear transformations learned from behavioral responses.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across three datasets, alignment varies more with training objective and pretraining data than with architecture or model size, and larger, more diverse datasets align better than ImageNet alone.

    Source: §1 · p. 2, §4.2 · p. 6, §5 · p. 9

  • Takeaways & Limitations

    Because image/text and JFT models with datasets ranging from 400M to 6.6B images have similar alignment, further dataset scaling alone may not close the gap; additional supervision may be needed.

    Source: §5 · pp. 9–10

    The study did not consider nonlinear transformations, which could potentially improve alignment, and its pretrained models vary in objectives and regularization strategies.

    Source: §5 · p. 9

Abstract

from arXiv · show

Today's computer vision models achieve human or near-human level performance across a wide variety of vision tasks. However, their architectures, data, and learning algorithms differ in numerous ways from those that give rise to human vision. In this paper, we investigate the factors that affect the alignment between the representations learned by neural networks and human mental representations inferred from behavioral responses. We find that model scale and architecture have essentially no effect on the alignment with human behavioral responses, whereas the training dataset and objective function both have a much larger impact. These findings are consistent across three datasets of human similarity judgments collected using two different tasks. Linear transformations of neural network representations learned from behavioral responses from one dataset substantially improve alignment with human similarity judgments on the other two datasets. In addition, we find that some human concepts such as food and animals are well-represented by neural networks whereas others such as royal or sports-related objects are not. Overall, although models trained on larger, more diverse datasets achieve better alignment with humans than models trained on ImageNet alone, our results indicate that scaling alone is unlikely to be sufficient to train neural networks with conceptual representations that match those used by humans.

1 INTRODUCTION

The paper asks whether neural networks’ conceptual representations align with human similarity judgments, addressing a question left unclear as vision research shifted toward practical applications. Across three behavioral datasets, it finds alignment depends more on training objectives and data than on model scale or architecture, and varies across human concepts.

1 INTRODUCTION

  • Although vision models have achieved strong benchmark performance, the extent to which their conceptual representations align with humans remains unclear.The shift toward practical applications has moved representation-learning research away from its similarities to human cognition.Source: §1 · p. 1
  • The study directly compares neural-network representations with human mental representations inferred from behavioral responses, focusing on similarity judgments from triplet and multi-arrangement tasks.Source: §1 · p. 2
  • Scaling ImageNet models does not consistently improve alignment with human similarity judgments; objective functions and other hyperparameters matter more than architecture or width/depth.Source: §1 · p. 2
  • Models trained on image/text data or larger, more diverse classification datasets than ImageNet achieve substantially better alignment with humans.Source: §1 · p. 2
  • A linear transformation trained on THINGS behavioral responses substantially improves alignment on held-out THINGS images and two independent multi-arrangement datasets.Source: §1 · p. 2
  • Food- and animal-related concepts are well represented, whereas sports-related and royal-object dimensions show weak human alignment, especially in ImageNet models.The analysis uses a sparse Bayesian model of human mental representations to partition triplets by the concept distinguishing the odd-one-out.Source: §1 · p. 2

2 RELATED WORK

Prior comparisons of neural networks with human behavior have often focused on classification errors, brain responses, or similarity judgments from smaller image and model sets. This paper’s related work also distinguishes semantic similarity judgments from perceptual similarity and situates human alignment among measures of model quality.

2 RELATED WORK

  • Networks trained on larger datasets show greater error consistency with humans, but ImageNet models differ from humans on classification errors, distorted images, and reliance on texture versus shape cues.Texture reliance can be mitigated through data augmentation, adversarial training, or larger datasets.Source: §2 · p. 2
  • Earlier semantic-similarity studies generally used fewer images or models, while Marjieh et al. (2022) reported only a weak parameter-count correlation and did not systematically examine alignment factors.Other studies examined similarity judgments for object sets, contextual associations, or ImageNet triplets.Source: §2 · p. 2
  • Unlike perceptual-similarity work, which favors intermediate layers of small architectures, this study finds best alignment with odd-one-out judgments at final layers and little architectural impact.Perceptual similarity compares a reference image with a distorted version; this study compares distinct images.Source: §2 · p. 2
  • Prior work found artificial and biological representations distinguish an odd-one-out that differs in category, but not natural images from synthetic scrambled images.Source: §2 · p. 3
  • Whereas pretraining accuracy often predicts out-of-distribution or transfer performance, objective and hyperparameter differences can affect transfer; here, better-performing architectures do not achieve greater human alignment.Source: §2 · p. 3

3 METHODS

The methods compare neural-network representations with human similarity judgments using THINGS triplets and two multi-arrangement datasets. Alignment is evaluated without adaptation, after linear probing, and with representational similarity analysis across a broad set of pretrained models.

3.1 DATA

  • THINGS provides 1,854 object categories and human triplet judgments; two multi-arrangement datasets add judgments for ImageNet images and other natural-image sets.Hebart et al. (2020) collected 1.46 million responses from 5,301 workers, with an estimated maximum odd-one-out accuracy of 67.22% ± 1.04%; Figure 1 illustrates a triplet where network and human choices differ.Source: §3.1 · p. 3, Figure 1 · p. 3

3.2 METRICS

  • Zero-shot accuracy identifies the closest pair by cosine similarity and scores whether the remaining image matches the human odd-one-out; linear probing learns a transformation from triplet responses, while RSA correlates human and model similarity matrices.The probe maximizes triplet-response log-likelihood with ℓ2 regularization, and alignment on multi-arrangement data is measured with Spearman correlation between RSMs.Source: §3.2 · p. 4

3.3 MODELS

  • The evaluation spans supervised and self-supervised ImageNet models, EcoSet and JFT models, and image/text models; VICE supplies interpretable human concept representations for analysis.VICE is an approximate Bayesian method for explaining human triplet responses, but it is not a vision model and predicts only judgments for images in its training triplets.Source: §3.3 · pp. 4–5

4 EXPERIMENTS

Across triplet and multi-arrangement similarity judgments, alignment tracks training objective and data more than architecture, scale, or ImageNet accuracy. Linear probing improves cross-dataset alignment, while concept-level performance varies across human dimensions.

4 EXPERIMENTS

  • The experiments compare neural-network representation spaces with human concept spaces and test whether concepts can be recovered through linear transformation.Odd-one-out accuracy is measured on THINGS unless otherwise stated.Source: §4 · p. 5

4.1 ODD-ONE-OUT VS. IMAGENET ACCURACY

  • ImageNet accuracy correlates only weakly with human odd-one-out accuracy (r = 0.099), and model parameter count shows no relationship with odd-one-out accuracy.Source: §4.1 · pp. 5–6
  • Training objective affects alignment more than architecture; among Siamese self-supervised models, contrastive methods outperform non-contrastive methods despite reversed ImageNet performance.Source: §4.1 · p. 6

4.2 CONSISTENCY OF RESULTS ACROSS DIFFERENT DATASETS

  • Human alignment patterns replicate across the triplet and multi-arrangement datasets: objective and pretraining data matter, but architecture and parameter count do not.Source: §4.2 · p. 6

4.3 HOW MUCH ALIGNMENT CAN A LINEAR PROBE RECOVER?

  • Linear probing substantially improves alignment across all three datasets; transformed image/text and ViT-G/14 JFT models outperform ImageNet and EcoSet models.Source: §4.3 · p. 6

4.4 HOW WELL DO PRETRAINED NEURAL NETS REPRESENT HUMAN CONCEPTS?

  • VICE-based concept analysis filters triplets to those predicted correctly by VICE, then partitions them into 45 subsets according to the dimension distinguishing the similar pair.Source: §4.4 · pp. 7–8

4.4.1 HUMAN ALIGNMENT IS CONCEPT-SPECIFIC

  • Concept rankings vary: ImageNet models are weakest on an animal dimension, while image/text and ViT-G/14 JFT lead most probed concepts, especially sports-related objects.For vehicle concepts, ResNets trained with cosine softmax are the best zero-shot models, while image/text models are among the worst.Source: §4.4.1 · p. 8

4.4.2 CAN HUMAN CONCEPTS BE RECOVERED VIA LINEAR REGRESSION?

  • Leading VICE dimensions achieve R2 scores above 0.7 for most models, but regression fit declines as dimension importance decreases.Image/text models and ViT-G/14 JFT provide the best fit, while AlexNet and EfficientNets perform poorest.Source: §4.4.2 · p. 8

5 DISCUSSION

Across the three investigated datasets, alignment depends more on training objectives and data than on ImageNet model scale, architecture, or capacity. Some human-relevant concepts are recoverable from network representations, while sports and royal concepts are harder to recover.

5 DISCUSSION

  • The study did not test nonlinear transformations, and pretrained models vary in objectives and regularization, though controlled comparisons partly mitigate the latter concern.The authors note that simple nonlinear transformations might improve alignment.Source: §5 · p. 9
  • Across all three datasets, ImageNet model scale does not improve alignment; objective function matters more, while architecture and capacity are insignificant.Models trained on image/text or more diverse data align better than ImageNet models.Source: §5 · p. 9
  • Food- and animal-related concepts are easier to recover from network representations than sports-related or royal-object concepts.The difficulty is especially noted for ImageNet models.Source: §5 · p. 9
  • Image/text and JFT models reach 60–61.5% probing accuracy, versus 64% for VICE and 67% for a Bayes-optimal classifier.Because image/text models trained on datasets ranging from 400M to 6.6B images achieve similar alignment, the authors suspect further scaling alone is unlikely to close the gap; additional supervision may be needed.Source: §5 · pp. 9–10

A EXPERIMENTAL DETAILS

The experimental appendices describe object-disjoint cross-validation for linear probing, temperature scaling for model probabilities, and nested cross-validation for regression. They also document the evaluated model families and choices of layer and similarity measure.

A.1 LINEAR PROBING

  • Object-based k-fold cross-validation keeps each triplet’s objects together in train or test, discarding triplets that span both sets; the final linear-probing experiments use 3-fold CV.The resulting sets contain about 433k training and 54k test triplets; regularization strength was grid-searched per model and fold setting.Source: §A.1 · pp. 18–19

A.2 TEMPERATURE SCALING

  • Temperature scaling multiplies the similarity matrix by τ, and expected calibration error provided the clearest optimum among the three criteria considered for choosing τ.The alternatives were average Jensen–Shannon distance and Kullback–Leibler divergence against VICE probabilities; ECE compares accuracy and confidence across subsets.Source: §A.2 · p. 19

A.3 LINEAR REGRESSION

  • VICE-dimension regression uses ridge regression and nested cross-validation, with 5-fold outer CV for the final R² estimates and leave-one-out inner CV to select α.The inner procedure grid-searches α.Source: §A.3 · pp. 19–20

B MODELS

  • The evaluated models span supervised ImageNet, image/text, self-supervised, and JFT-trained families; later layers generally perform better, and cosine similarity generally aligns better than dot product.Analyses therefore use logits or penultimate/embedding layers; self-supervised models use ResNet-50 architectures.Source: B MODELS · pp. 20, 22

C CIFAR-100 TRIPLET TASK

The CIFAR-100 coarse triplet task tests whether findings from human-alignment comparisons depend on the THINGS task. ImageNet accuracy correlates strongly with accuracy on this task, unlike its weaker relationship with human odd-one-out accuracy.

C CIFAR-100 TRIPLET TASK

  • The task uses 50,000 triplets formed from two images in one CIFAR-100 coarse class and an odd-one-out image from another.Images are sampled from the training set, excluding validation examples.Source: C CIFAR-100 TRIPLET TASK · p. 22
  • ImageNet accuracy correlates with CIFAR-100 coarse odd-one-out accuracy at r = 0.70, unlike its significantly weaker correlation with human odd-one-out accuracy.The plotted diagonal is a least-squares fit.Source: C CIFAR-100 TRIPLET TASK · p. 22, Figure C.2 · p. 22
  • The coarse-class task was designed to test whether THINGS-task findings reflect the triplet task itself, using superclasses considered potentially comparable to higher-level human concepts.The authors note that human triplet judgments use sparse concept representations, as do CIFAR-100 coarse superclasses.Source: C CIFAR-100 TRIPLET TASK · p. 22

D TRANSFERABILITY OF PENULTIMATE LAYER VS. LOGITS

The appendix compares penultimate-layer embeddings and logits as representations for transfer learning. Across nearly all models and transfer datasets, penultimate-layer embeddings provide better transfer accuracy.

D TRANSFERABILITY OF PENULTIMATE LAYER VS. LOGITS

  • The comparison evaluates ℓ2-regularized multinomial logistic-regression classifiers on 12 transfer datasets using representations from 16 models.The datasets include Food-101, CIFAR-10 and CIFAR-100, Birdsnap, SUN397, Stanford Cars, FGVC Aircraft, PASCAL VOC 2007, DTD, Oxford-IIIT Pets, Caltech-101, and Oxford 102 Flowers.Source: D TRANSFERABILITY OF PENULTIMATE LAYER… · p. 23
  • Penultimate-layer embeddings provide better linear-transfer representations than logits for nearly all models and datasets.Figure D.1 compares the two layers across 16 models using multinomial logistic-regression accuracy; the models were trained with vanilla softmax cross-entropy.Source: Figure D.1 · p. 23, D TRANSFERABILITY OF PENULTIMATE LAYER… · p. 23

E LINEAR PROBING

Linear probing performance closely tracks zero-shot odd-one-out accuracy, but probing alignment varies little with ImageNet accuracy, architecture, or model size. Training objective matters more, and the penultimate layer usually aligns best after applying the learned transformation.

E LINEAR PROBING

  • Probing and zero-shot odd-one-out accuracies correlate strongly (r = 0.963) for ImageNet-1K/21K models.Figure E.1 compares the two measures.Source: E LINEAR PROBING · p. 24, Figure E.1 · p. 24
  • Probing accuracy has low correlation with ImageNet accuracy (r = 0.213), while architecture and model size do not affect odd-one-out accuracy.Figure E.3 shows probing accuracy against ImageNet accuracy and model parameters; Figure E.4 reports the pattern across both multi-arrangement datasets.Source: E LINEAR PROBING · p. 24, Figure E.3 · p. 25, Figure E.4 · p. 26
  • Objective-function differences explain more alignment variation than architecture or model size; probing reduces differences among loss functions and Siamese SSL models, though Siamese models remain ahead of non-Siamese models.Cosine softmax remains among the strongest models after probing.Source: E LINEAR PROBING · p. 24, Figure E.4 · p. 26
  • After applying the learned transformation W, alignment is nearly always highest for the penultimate layer across odd-one-out and RSA evaluations.Figure E.2 compares logits and penultimate layers across all three datasets, before and after transformation.Source: E LINEAR PROBING · p. 24, Figure E.2 · p. 25

F LINEAR REGRESSION

Linear regression fits neural representations to VICE dimensions and evaluates held-out fit and transformed odd-one-out accuracy. Regression and probing results are strongly related, while the entropy analysis finds no model strongly aligned with VICE and a role for objective function in entropy alignment.

F LINEAR REGRESSION

  • The regression fits each of 45 VICE dimensions separately from neural representations, selects its regularization hyperparameter by cross-validation, and evaluates held-out R2 and transformed odd-one-out accuracy.Figure F.4 reports R2 scores for the embedding-layer fits.Source: F LINEAR REGRESSION · p. 27, Figure F.4 · p. 28
  • Leading VICE dimensions achieve R2 > 0.7 for most models, and regression odd-one-out accuracy closely tracks probing for embedding (r = 0.982) and logit (r = 0.972) layers.Fit quality generally declines as dimension importance decreases; Figure F.1 and Figure F.3 compare regression with probing.Source: F LINEAR REGRESSION · p. 27, Figure F.1 · p. 27, Figure F.3 · p. 28
  • Regression odd-one-out accuracy relates strongly to zero-shot accuracy (r = 0.795), and logit-versus-penultimate-layer results agree with the linear-probing findings.Figure F.2 compares zero-shot and regression accuracy and regression accuracy across layers.Source: F LINEAR REGRESSION · p. 27, Figure F.2 · p. 27
  • Linear-regression odd-one-out accuracy is slightly higher than probing, potentially because VICE is trained on all images and the regression transform indirectly accesses evaluation images.The linear probe does not access those images.Source: F LINEAR REGRESSION · p. 27

G ENTROPY

  • No model has a strong positive correlation with VICE triplet entropy, although objective function appears important to entropy alignment among models and with VICE.Models with the same architecture often correlate strongly in their uncertainty across triplets.Source: G ENTROPY · p. 28

G.1 HUMAN UNCERTAINTY DETERMINES ODD-ONE-OUT ERRORS

The analysis bins THINGS triplets by VICE-estimated human response entropy and compares model errors across those bins. Errors rise with entropy, but model classes differ sharply on low-entropy triplets.

G.1 HUMAN UNCERTAINTY DETERMINES ODD-ONE-OUT ERRORS

  • ImageNet-trained VGGs, ResNets, EfficientNets, ViTs, and SSL models have low-entropy triplet errors of 0.1–0.3, while ALIGN, CLIP, BASIC-L, and ViT-G/14 JFT are near zero.Figure G.3 compares errors across triplet-entropy bins and model classes.Source: §G.1 · p. 29, Figure G.3 · p. 30
  • All models’ zero-shot odd-one-out error rates increase monotonically as VICE-estimated human triplet entropy increases.Triplets are partitioned into 11 entropy bins; entropy is estimated from 50 Monte Carlo samples.Source: §G.1 · pp. 28–29

H DIFFERENT CONCEPTS ARE DIFFERENTLY DISENTANGLED IN DIFFERENT

The visualizations show differences in how higher-level concepts are disentangled across model representations. Separately, model pairs’ odd-one-out choice agreement is highly correlated with their linear CKA, suggesting that choice agreement reflects representational similarity.

H DIFFERENT CONCEPTS ARE DIFFERENTLY DISENTANGLED IN DIFFERENT

  • Animal concepts appear more disentangled from other higher-level categories in CLIP image/text representations than in ImageNet models, while Jigsaw concepts appear less disentangled than in SimCLR and BarlowTwins.Figure H.1 uses t-SNE visualizations of selected models’ embedding layers, with points labeled by THINGS categories.Source: H DIFFERENT CONCEPTS ARE DIFFERENTLY… · p. 31, Figure H.1 · p. 31

I ODD-ONE-OUT AGREEMENTS REFLECT REPRESENTATIONAL SIMILARITY

  • Linear CKA compares centered representations of the same examples through their representational similarity matrices.The matrices encode example-to-example similarity; Figure I.1 compares CKA with odd-one-out choice agreement.Source: I ODD-ONE-OUT AGREEMENTS REFLECT… · p. 32, Figure I.1 · p. 32
  • Odd-one-out choice agreement correlates highly with linear CKA for nearly every model pair, indicating that behavioral choice agreement reflects representational similarity.Figure I.1 shows agreement and CKA heatmaps for the same model pairs and plots agreement against CKA.Source: I ODD-ONE-OUT AGREEMENTS REFLECT… · p. 32, Figure I.1 · p. 32

J HUMAN ALIGNMENT IS CONCEPT SPECIFIC

Human alignment varies by concept and by evaluation method: image/text models and ViT-G/14 JFT usually lead in probing, but some concepts favor ImageNet models in zero-shot comparisons. Royal and sports-related dimensions show the largest alignment gap, while more important concepts are generally easier to recover.

J HUMAN ALIGNMENT IS CONCEPT SPECIFIC

Loading 2211.01201v5…