Source-linked AI summary
Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition
Saeed Reza Kheradpisheh, Masoud Ghodrati, Mohammad Ganjtabesh, Timothée Masquelier
TL;DR
The paper asks whether DCNNs match humans in view-invariant object recognition, including their errors and representations, as viewpoint variation changes. It benchmarks multiple model types against masked human responses using controlled variations and finds that depth matters mainly for larger variations, with an 18-layer network outperforming humans at the highest level.
Problem
It was unknown whether DCNNs match human performance, errors, and representations in view-invariant object recognition across different magnitudes of viewpoint variation.
Method
The study benchmarked eight DCNNs, HMAX, and a shallow pixel model against humans using controlled object variations, backward masking, error comparisons, and representational analyses.
Results
Deeper models were more human-like mainly under large variations; some reached human performance, and an 18-layer network outperformed humans at the highest variation level.
Takeaways & Limitations
Model depth is most relevant for human-like invariant recognition when viewpoint variations are large, while representational similarity need not track top performance.
Abstract
from arXiv · showhide
Deep convolutional neural networks (DCNNs) have attracted much attention recently, and have shown to be able to recognize thousands of object categories in natural image databases. Their architecture is somewhat similar to that of the human visual system: both use restricted receptive fields, and a hierarchy of layers which progressively extract more and more abstracted features. Yet it is unknown whether DCNNs match human performance at the task of view-invariant object recognition, whether they make similar errors and use similar representations for this task, and whether the answers depend on the magnitude of the viewpoint variations. To investigate these issues, we benchmarked eight state-of-the-art DCNNs, the HMAX model, and a baseline shallow model and compared their results to those of humans with backward masking. Unlike in all previous DCNN studies, we carefully controlled the magnitude of the viewpoint variations to demonstrate that shallow nets can outperform deep nets and humans when variations are weak. When facing larger variations, however, more layers were needed to match human performance and error distributions, and to have representations that are consistent with human behavior. A very deep net with 18 layers even outperformed humans at the highest variation level, using the most human-like representations.
Introduction
The study tests whether DCNNs match humans in view-invariant object recognition when viewpoint variation is carefully controlled. It compares performance, errors, and representations across models and humans under feed-forward-biased conditions.
- DCNNs were developed to model the hierarchical, feed-forward organization of primate ventral visual processing.
- Previous studies often evaluated difficult image databases without carefully controlling viewpoint variation, limiting conclusions about invariant recognition.
- The study systematically compared humans with eight DCNNs, HMAX, and a pixel-based shallow model using the same images.
- Experiments varied object size, position, and 2-D and 3-D rotations, while also examining uniform and natural backgrounds.
- Deeper models were usually more human-like only with large variations, and some models matched human performance, errors, or representations.
Materials and methods
The evaluation used multiple pretrained DCNN architectures, HMAX, and a pixel baseline on systematically varied rendered objects. Human comparisons used brief masked presentations, while model and human representations were compared through error distributions and RDMs.
- The benchmark included eight supervised DCNNs spanning different architectures and depths, alongside HMAX and a linear SVM using pixel values.
- Model selection: The 19-layer Very Deep model was included to examine how increased depth affects recognition accuracy.
- Image generation: The object database contained five categories and seven variation levels spanning size, position, in-depth rotation, in-plane rotation, and background.
- Psychophysical experiments: Human observers viewed briefly presented objects followed by a 100 ms noise mask intended to minimize feedback effects.
- Representational dissimilarity matrix (RDM): RDMs represented pairwise response dissimilarities and compared model representations with human behavioral response patterns.
Results
DCNN performance depended on variation magnitude and background context: shallow or early-layer representations could suffice for weak variation, whereas larger variations favored deeper processing. CNN-F, CNN-M, and CNN-S most consistently reached human-level accuracy, while error distributions and representational similarity provided additional distinctions among models.
- Accuracy across variation levels: At the most difficult uniform-background level, CNN-S, CNN-M, and Zeiler and Fergus did not differ significantly from humans, with CNN-S the best model.The comparison used level 7 and Wilcoxon rank-sum tests.
- Background effects: Natural backgrounds reduced accuracy for both humans and models; at level 7, CNN-S and CNN-M remained statistically comparable to humans, while other listed models fell significantly below them.Only three DCNNs performed close to human overall in the natural-background condition, and the human advantage over CNN-F, Zeiler and Fergus, Krizhevsky, Hybrid-CNN, and Overfeat was significant at level 7.
- Accuracy across variation levels: At low variation levels, nearly all DCNNs and even pixel representations achieved perfect or near-perfect accuracy with uniform backgrounds.The task was described as simple under small variations and uniform gray backgrounds.
- Error distributions: HMAX and pixel representations deteriorated sharply at high variation, while DCNNs retained clearer categorical information and made fewer errors across categories.Human errors were concentrated mainly in ship–airplane confusions, whereas model error distributions differed by architecture and category.
- Accuracy across layers: Across layers, later processing generally helped under larger variations, but natural backgrounds made deeper processing non-monotonic: layer 2 was often best for DCNNs, whereas only the last three layers reached human-level accuracy at intermediate variation.For uniform backgrounds, the final layers were especially important at difficult levels; for natural backgrounds, the last convolutional layer often outperformed fully connected layers.
Discussions
DCNN performance and human-likeness depend strongly on viewpoint variation and network depth. Deep models can match or exceed humans under large variations, whereas shallow models may perform better when variations are weak, with feedback and contextual processing remaining important limitations.
- Evaluation: The study systematically compared humans, eight DCNNs, HMAX, and shallow baselines using identical images while controlling viewpoint-variation magnitude.It also compared accuracy, error distributions, representational geometry, and layer-specific changes in invariance.
- Variation and depth: Deeper networks were usually better and more human-like only when object variations were large.Under weak variations, shallow models outperformed deeper models and humans; increasing variation disturbed representations and increased misclassification, especially in lower layers.
- Variation and depth: The Very Deep CNN with 19 layers significantly outperformed all other models and humans, while invariance increased progressively across its layers at high variation levels.This result is consistent with the paper’s conclusion that deeper networks can reach or exceed human feed-forward vision accuracy in difficult invariant-recognition conditions.
- Limitations and future directions: DCNNs remain constrained by missing biological mechanisms, including feedback, figure-ground segregation, and attention, which may matter especially for natural or cluttered backgrounds.The psychophysical masking procedure weakened but did not completely eliminate feedback effects in humans, while adding feedback to DCNNs could increase performance at complex variation levels with greater computational cost.