Source-linked AI summary
Linguistic generalization and compositionality in modern artificial neural networks
Marco Baroni
TL;DR
The paper asks whether deep networks achieve linguistic productivity through systematic compositional rules. It reviews modern architectures and studies of grammatical and compositional generalization, finding abstract grammar-dependent behavior alongside failures on systematic compositional tests. These results motivate studying neural networks as alternative computational systems for linguistic productivity.
Problem
Deep networks generalize impressively in language, raising whether their productivity depends on the algebraic compositional rules often attributed to human language.
Method
The paper reviews modern language-processing architectures and synthesizes experiments testing grammatical generalization and compositionality in neural networks.
Results
Networks capture structure-dependent grammatical generalizations, yet recurrent models achieve 100% accuracy on random SCAN splits while reaching only 12.5% on jump and 2.5% on around-right splits requiring systematic composition.
Takeaways & Limitations
Neural networks may handle linguistic productivity through strategies beyond rule-based compositionality, offering a comparative perspective for linguistics and cognitive science.
Takeaways & Limitations
The reviewed grammatical-generalization evidence does not establish that the networks possess a compositional rule-based system, and current models remain brittle to adversarial examples.
Abstract
from arXiv · showhide
In the last decade, deep artificial neural networks have achieved astounding performance in many natural language processing tasks. Given the high productivity of language, these models must possess effective generalization abilities. It is widely assumed that humans handle linguistic productivity by means of algebraic compositional rules: Are deep networks similarly compositional? After reviewing the main innovations characterizing current deep language processing networks, I discuss a set of studies suggesting that deep networks are capable of subtle grammar-dependent generalizations, but also that they do not rely on systematic compositional rules. I argue that the intriguing behaviour of these devices (still awaiting a full understanding) should be of interest to linguists and cognitive scientists, as it offers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality, and it might lead to new insights into the less systematic generalization patterns that also appear in natural language.
1 Introduction
Modern neural networks achieve strong language performance, motivating cognitive-scientific study despite their optimization for practical goals rather than human-like cognition. The paper examines whether their productivity depends on compositional rules.
- Deep networks now support effective machine translation and generate coherent novel text, demonstrating substantial natural-language capability.
- Human language is highly productive, and compositionality is commonly treated as the mechanism combining simpler parts into larger expressions.
- The paper adopts a broad notion of compositionality that includes syntactic rules for judging novel sentences, not only semantic composition.
- Compositional operations are often understood as systematic algebraic functions that apply consistently across variables of the right type.
- The paper reviews modern architectures, presents evidence for subtle syntactic generalization, and argues that these systems lack convincing rule-based compositionality.
2 Modern deep networks for language processing: what has changed
Modern language networks combine large-scale data-driven training with architectural mechanisms that regulate information flow, separate input from output, and selectively retrieve encoded states. These changes support flexible sequence processing while embedding strong structural priors.
- Larger datasets, computational power, and improved optimization enabled large-scale training of complex multilayer networks.
- Language modeling trains a sequence network to predict the next word from preceding context and is widely used for linguistic pretraining.
- Gated recurrent networks learn mechanisms that regulate information retention, updating, and contribution to the current output across time.
- Encoder-decoder architectures use separate subnetworks for processing inputs and generating outputs, enabling sequence-to-sequence mappings.
- Attention mechanisms let decoders retrieve different amounts of information from stored encoder states during generation.
- These networks are not tabulae rasae: gates, encoding-decoding modules, and attention provide strong structural priors unlike those traditionally proposed for human linguistic competence.
3 Colorless green grammatical generalization in deep networks
Controlled experiments test whether language networks generalize grammatical structure rather than memorized patterns. The results indicate above-chance sensitivity to abstract agreement, with circuit analyses supporting genuine grammatical processing but not establishing compositional rules.
- Gulordava and colleagues test gated recurrent networks on semi-random nonsense minimal pairs requiring long-distance number agreement.
- The nonsense stimuli remove semantic, lexical, and collocational confounds, focusing the evaluation on abstract grammatical generalization.
- Across English, Hebrew, Italian, and Russian, networks preferred grammatical sentences above chance and competitive baselines; English accuracy was 74% versus 50% chance.
- In Italian, human accuracy was only marginally higher than network accuracy, at 88.4% versus 85.5%.
- Ablation studies found a small set of units carrying long-distance number information and connected to nodes sensitive to hierarchical constituency.
- The evidence concerns grammatical productivity, but it does not yet establish that the network has a compositional rule-based system.
4 Compositional generalization: can deep networks dax twice?
SCAN tests whether sequence-processing networks generalize commands compositionally by withholding familiar primitives from composite contexts. Recurrent networks succeed on generic productivity but fail systematic splits, while an attention-based convolutional model achieves partial success without perfect rule-like generalization.
- SCAN benchmark: SCAN maps navigation commands compositionally into action sequences, such as “jump twice and run” becoming JUMP JUMP RUN.The benchmark combines primitive verbs with modifiers and conjunctions under an explicit interpretation function.
- Evaluation design: Networks are trained on some commands and evaluated on unseen commands, with partition design revealing which generalizations they learned.The random split tests generic productivity, whereas specialized splits test compositional generalization.
- Compositional splits: The jump split tests whether a primitive learned only in isolation transfers to composite contexts such as “jump twice” and “walk and jump”.This transfer is straightforward for a system applying a rule like “x twice” → [[x]] [[x]].
- Results: Recurrent networks reach 100% accuracy on the random split but only 12.5% on jump and 2.5% on around-right.The reported standard deviations are approximately 0.0%, 6.6%, and 2.7%, respectively, across multiple runs.
- Results: An attention-based convolutional network retains 100% random-split accuracy and reaches 69.2% on jump and 56.7% on around-right, indicating partial but imperfect generalization.The corresponding standard deviations are 0.0%, 8.2%, and 10.2%; chance accuracy is practically 0%.
- Related evidence: Communication-game agents likewise show that low-compositionality codes can generalize as well as more compositional codes.Across random initializations, sender agents developed codes with varying degrees of compositionality.
5 Conclusion
Modern deep networks can display productive, structure-dependent language behavior without relying on systematic compositional rules. This dissociation motivates both better analysis of their mechanisms and comparison with fuzzier productive patterns in natural language.
- Conclusion: Deep networks can be highly proficient in language while lacking anything resembling systematic compositional rules.The reviewed evidence extends beyond shallow pattern matching to structure-dependent generalizations traditionally associated with compositional rules.
- Open questions: The mechanisms behind the dissociation between productive grammatical competence and systematic compositionality remain poorly understood.The paper identifies better analytical tools as a priority for uncovering the strategies learned by these networks.
- AI implications: A central AI question is whether more compositional architectures can improve adaptivity and learning speed without sacrificing generality.Structured modular architectures are proposed as one possible route, with potential implications for robustness and difficult NLP tasks.
- Linguistic implications: Neural-network generalization offers a comparative perspective on productive linguistic phenomena governed by less systematic, fuzzier laws.Examples include irregular-inflection patterns, partially transparent derivational morphology, semi-lexicalized syntax, and early grammaticalization.