Deep Learning: A Critical Appraisal

Gary Marcus

Source-linked AI summary

Deep Learning: A Critical Appraisal

Gary Marcus

arXiv:1801.00631v1Jan 2, 2018cs.AIcs.LGstat.ML

TL;DR

The paper examines whether deep learning’s recent successes reveal a general route to intelligence, or expose important limits. It critically assesses the approach across capabilities and failure modes, concluding that major advances coexist with unresolved generalization problems and that additional techniques may be needed for artificial general intelligence.

Problem
The paper addresses what deep learning reveals about intelligence, what it can be expected to do, where it may break down, and how close the field is to artificial general intelligence.
Method
The paper presents ten concerns for deep learning by reviewing its strengths, weaknesses, capability misunderstandings, and possible directions forward.
Results
Major advances in speech recognition, machine translation, and board games coexist with persistent concerns about deep learning’s limitations.
Takeaways & Limitations
There is no general deep-learning solution for generalizing outside the training space, so different kinds of solutions are needed to reach artificial general intelligence.
Takeaways & Limitations
Deep learning works best in largely stable worlds and can be problematic in changing systems such as politics, economics, or stock prediction.

Abstract

from arXiv · show

Although deep learning has historical roots going back decades, neither the term "deep learning" nor the approach was popular just over five years ago, when the field was reignited by papers such as Krizhevsky, Sutskever and Hinton's now classic (2012) deep network model of Imagenet. What has the field discovered in the five subsequent years? Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence.

1. Is deep learning approaching a wall?

Deep learning has driven major advances and broad enthusiasm, but the paper argues that its progress may be approaching a wall. It examines what deep learning can do, where it breaks down, and what may be needed for artificial general intelligence.

2012 ImageNet results helped make deep learning practical and rapidly transformed its visibility in artificial intelligence.The paper attributes this shift partly to increases in computational power and data.
Deep learning has produced state-of-the-art results in speech recognition, image recognition, language translation, and many current AI applications.
The paper asks whether deep learning is approaching a wall despite its successes and growing public enthusiasm.
Its aims are to temper irrational exuberance, assess deep learning’s weaknesses, and consider how the field might move forward.
Five years into the resurgence, the paper presents critical reflection on what deep learning has and has not achieved.

2. What deep learning is, and what it does well

Deep learning primarily learns statistical input-output mappings through multilayer neural networks, often using convolution to exploit regularities such as translational invariance. Its applications include classification and other forms of prediction, generation, compression, and inference.

Deep learning is primarily a statistical technique for classifying patterns from sample data with multilayer neural networks.
Beyond classification, deep learning can support regression, generative models, image compression, probability-distribution learning, and variational inference.
A typical neural network contains input units, multiple hidden layers, output units, and learned connections adjusted by back-propagation and gradient descent.
Convolution constrains neural connections to capture translational invariance, allowing objects to retain their identity across image locations.
Deep learning learns mappings such as speech sounds to words, images to labels, and game pixels to joystick positions.

3. Limits on the scope of deep learning

The paper argues that deep learning’s strengths depend on abundant data and similarity between training and test examples. Its limitations become most consequential when systems must learn abstractions, extrapolate beyond training space, or operate outside stable, finite domains.

Deep learning must often generalize beyond observed data, but finite data limit formal guarantees of high-quality performance.
Generalization includes interpolation between known examples and extrapolation beyond the space of training examples.
Neural networks generally need large datasets and test examples similar to training data; ImageNet used roughly a million examples across approximately one thousand categories.
The paper presents ten challenges faced by current deep learning systems, including learning abstract relationships from few examples.
Deep learning lacks a mechanism for learning abstractions through explicit verbal definition and works best with thousands, millions, or billions of examples.
Convolutional networks may face exponential inefficiencies when generalizing to novel viewpoints, requiring replicated detectors or exponentially larger labeled datasets.

3.2.Deep learning thus far is shallow and has limited capacity for transfer

The paper argues that deep architectures do not necessarily produce conceptual understanding or robust transfer. Systems can perform impressively on familiar scenarios yet rely on superficial contingencies that fail under small changes.

“Deep” refers to many hidden layers, not conceptual depth or natural representations of abstract concepts such as justice or democracy.
Atari systems can match human experts across games while lacking robust concepts such as walls or opponents.
Breakout transfer tests showed failures after minor perturbations, including moving the paddle’s height or inserting a midscreen wall.
Comparable adversarial examples undermined DQN, A3C, and related game-playing techniques across several games.
75% to 36%: across sixteen models, question-answering accuracy dropped sharply after a changed example.
The paper concludes that patterns extracted by deep learning are often more superficial than they initially appear.

3.3.Deep learning thus far has no natural way to deal with hierarchical structure

Deep learning systems do not naturally represent hierarchical structure, limiting systematic generalization and inference when novel situations require compositional or nested relationships.

Current deep-learning models represent sentences largely as sequences rather than explicitly hierarchical structures such as syntactic trees.
RNNs generalize well when training and test differences are small but fail spectacularly when generalization requires systematic compositional skills.
This limitation affects planning and motor control when systems encounter novel situations requiring complex hierarchical structure.
Recurrent networks remain limited in representing and faithfully generalizing rich structure.
Weak structural representation also limits inference beyond information explicitly stated in text.

3.5.Deep learning thus far is not sufficiently transparent

Deep learning systems remain opaque because their learned parameters are not expressed in human-interpretable terms, creating practical concerns for debugging, diagnosis, finance, and bias.

Deep networks contain millions or billions of parameters whose meanings are identified by network location rather than human-interpretable labels.
Neural networks as a whole remain largely black boxes despite progress in visualizing individual node contributions.
Opacity can be crucial for debuggability when deep-learning systems are integrated into larger systems.
Unresolved transparency is a potential liability in financial trading and medical diagnosis, where users may need to understand decisions.
Deep learning typically learns input-output relations from training databases rather than explicitly incorporating prior knowledge such as physical laws.
Integrating prior knowledge is difficult because learned representations mainly encode opaque feature correlations rather than quantified or generic abstractions.
Open-ended real-world problems provide sporadic, unencapsulated data and require broader knowledge than neatly packaged benchmark tasks.

3.7.Deep learning thus far cannot inherently distinguish causation from correlation

Deep learning learns correlations between input and output features without an inherent representation of causality, so correlated variables need not reveal causal direction.

Deep learning learns complex input-output correlations but has no inherent representation of causality.
The correlation between height and vocabulary illustrates that association does not identify whether either variable causes the other.
Causality has been central to some other approaches to artificial intelligence.

3.8.Deep learning presumes a largely stable world, in ways that may be problematic

Deep learning is expected to work best in stable environments but less well in changing systems, where learned approximations can be brittle, difficult to engineer, and hard to guarantee.

Deep learning is likely to work best in stable worlds with unvarying rules and less well in changing systems such as politics and economics.
Deep-learning systems can perform well across much of a domain yet remain easily fooled by unusual inputs.
Vision systems have mistaken yellow-and-black stripes for school buses and sticker-covered parking signs for refrigerators.
Real-world stop signs have been classified as speed-limit signs, and 3D-printed turtles as rifles.
Other demonstrations include a police system confusing nudes with sand dunes.
Despite active research, no robust solution to deep-learning spoofability had been found four years after it was first noted.
Deep learning is difficult to engineer robustly because systems may work only in limited circumstances and lack comparable guarantees, incrementality, transparency, and debuggability.
Robustness and replicability remain serious issues in deep reinforcement learning, while automated system development still has a long way to go.

3.11. Discussion

Deep learning is effective for mapping inputs to outputs when data are abundant and test cases resemble training examples, but it remains weak at extrapolating beyond the training space. These limitations affect novelty, open-ended inference, and prospects for artificial general intelligence.

3.11. Discussion: Deep learning excels at closed-end classification when sufficient data are available and test examples closely resemble training data.Deviations from these assumptions create problems, especially with limited data, distribution shifts, and novelty.
3.11. Discussion: Three-layer perceptrons trained on even numbers learned interpolation but repeatedly failed to apply the identity function to odd numbers.For example, a network could produce f(15) = 14 despite correctly handling seen and nearby even numbers.
3.11. Discussion: Adding hidden units, hidden layers, or contextual pretraining did not solve the networks’ failure to generalize identity beyond their training space.The cited experiments found that odd numbers remained outside the learned training space.
3.11. Discussion: Contemporary neural networks still perform well near core training data but begin to break down farther into the periphery.The paper connects this persistent extrapolation problem to data hunger, fooling vulnerability, open-ended inference, and transfer.
3.11. Discussion: Convolution builds in translational invariance, but this addresses only one class of generalization problems rather than providing a general solution.Data augmentation can broaden the training-example space, but the paper still identifies no general deep-learning solution for extrapolation.

4. Potential risks of excessive hype

Excessive AI hype risks disappointing investors and users when systems fail to match broad claims, potentially triggering a downturn in popularity and funding. The paper also warns that the field could become intellectually trapped in a narrow local minimum.

4. Potential risks of excessive hype: Overhyping AI could contribute to another AI winter by raising expectations beyond systems’ brittleness, narrowness, and superficiality.The paper notes that hype remains a major concern despite AI’s many practical applications.
4. Potential risks of excessive hype: Executives investing heavily in AI may be disappointed, as illustrated by Facebook’s M project being downgraded to narrowly defined assistant tasks.The project was launched publicly as a general-purpose personal assistant before being reduced in scope.
4. Potential risks of excessive hype: If highly publicized systems such as driverless cars disappoint on safety or full autonomy, AI could suffer a sharp decline in popularity and funding.The paper also says chatbots generally have not lived up to the hype they received several years earlier.
4. Potential risks of excessive hype: The field may become trapped in a local minimum by concentrating on accessible but limited models and neglecting riskier research directions.The concern is intellectual overfocus on low-hanging fruit rather than only a temporary failure of one technique.
4. Potential risks of excessive hype: Avoiding stagnation matters because AI could contribute to automated scientific discovery and more sophisticated medicine.The paper frames these potential rewards alongside the risk that the field becomes stuck in a local minimum.

5. What would be better?

The paper argues that deep learning should remain part of AI but be combined with complementary approaches suited to inference, abstraction, knowledge, and autonomous problem solving. It proposes unsupervised learning, symbolic methods, human cognition, and challenge problems as directions beyond supervised data-intensive learning.

5. What would be better?: Deep learning should be treated as one tool among many rather than a universal solution.Its strengths in perceptual classification should be complemented by other techniques.
5. What would be better?: Deep learning is valuable for perceptual classification with abundant data but often less satisfactory in richer cognitive domains.The proposed research directions target capabilities that extend beyond this perceptual sweet spot.
5.1. Unsupervised learning: Unsupervised learning can reduce reliance on labeled data through clustering or prediction from successive video frames, but these approaches may remain data-hungry and lack explicit variables.The paper argues they do not inherently solve open-ended inference, interpretability, or debuggability.
5.3. Human cognition: Human cognitive and developmental psychology may provide mechanisms for more robust AI, including innate representations, abstract reasoning, autonomous goals, and narrative understanding.The paper recommends using psychological insights without requiring literal replication of the human mind.
5.2. Symbol-manipulation, and the need for hybrid models: Hybrid systems could combine deep learning’s perceptual classification with symbolic AI’s strengths in inference and abstraction.The paper cites neurosymbolic models, differentiable neural computers, differentiable interpreters, and neural programming as tentative steps.
5.4. Bolder challenges: Challenge problems should test comprehension, scientific reasoning, and transfer across games rather than only supervised learning on large datasets.These tasks require open-ended questions, inference beyond explicit statements, general knowledge, or transfer between substantially different games.

6. Conclusions

The paper concludes that major advances in speech, translation, and board games have not resolved deep learning’s broader limitations. It argues that reaching human-level cognitive flexibility will require tools beyond deep learning, while noting growing support for this view.

6. Conclusions: Deep learning still lacks clear mechanisms for causal relations, abstract concepts, logical inference, and integration of abstract knowledge.These concerns were raised earlier and remain valid despite advances in specific application domains.
6. Conclusions: Major advances in speech recognition, machine translation, and board games have not eliminated the paper’s central concerns.The paper contrasts these domain advances with continuing limitations in broader intelligence.
6. Conclusions: Scholars from multiple perspectives have increasingly emphasized similar limits of deep learning.The paper lists Lake, Baroni, Chollet, Jia, Liang, George, Abbeel, and colleagues among those raising related concerns.
6. Conclusions: Geoff Hinton expressed concern about back-propagation’s dependence on labeled datasets and suggested that entirely new methods may be needed.The paper presents this as a notable reconsideration by a pioneer of deep learning.