Source-linked AI summary
BERT Rediscovers the Classical NLP Pipeline
Ian Tenney, Dipanjan Das, Ellie Pavlick
TL;DR
The paper asks where BERT captures linguistically meaningful information and whether its internal organization resembles the traditional NLP pipeline. It applies edge probing and complementary layer analyses to BERT, finding an expected progression from lower-level syntax to higher-level semantics, while individual sentences can be processed dynamically.
Problem
It remains unclear whether high-performing pre-trained encoders learn linguistically meaningful abstractions or mainly model complex co-occurrence statistics.
Method
The study uses edge probing with scalar mixing weights and cumulative scoring across eight linguistic tasks to measure where information is encoded in BERT.
Results
BERT encodes tasks in a progression from POS and parsing through semantic roles to coreference, with syntactic information more localized and semantic information more distributed.
Takeaways & Limitations
The aggregate pipeline order coexists with sentence-level revision, as higher-level predicate-argument information can help disambiguate lower-level decisions.
Takeaways & Limitations
The findings are exploratory, focus on BERT, and probing observations do not establish how information is used.
Abstract
from arXiv · showhide
Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within the network. We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.
1 Introduction
Pre-trained encoders have improved NLP performance but remain difficult to interpret. This paper probes BERT to determine whether it captures recognizable linguistic abstractions and where those abstractions appear in the network.
- Pre-trained encoders improve NLP performance while raising questions about interpretability and whether they learn linguistically meaningful abstractions.
- Prior probing work studies model behavior and directly inspects networks for localized regions associated with linguistic decisions.
- The study uses probing tasks derived from the traditional NLP pipeline to quantify where specific linguistic information is encoded in BERT.
- The authors report that the order of encoded abstractions reflects the traditional hierarchy of NLP tasks.
2 Model
The study examines stock BERT models with edge probing to measure how linguistic information is distributed across layers. It evaluates eight labeling tasks using a common micro-averaged F1 metric.
- Edge probing tests how well linguistic structure can be extracted from pre-trained encoders using span representations and probing classifiers.
- The suite contains eight tasks spanning POS, constituents, dependencies, entities, semantic roles, coreference, semantic proto-roles, and relation classification.
- All tasks use micro-averaged F1, enabling comparison across the edge probing benchmarks.
- The experiments use stock uncased BERT base and large models pretrained with masked language modeling and next-sentence prediction on a 3.3B-word English corpus.
- A weighted sum across encoder layers produces per-token representations for task-specific probing classifiers.
- For single-span tasks, the classifier uses one span; POS represents that span as a single token.
- The datasets combine English Web Treebank, SPR1, SemEval 2010 Task 8, and OntoNotes 5.0 according to task.
- The study is exploratory and cautions that probing can miss information or reveal patterns without showing how BERT uses them.
3 Metrics
The paper introduces complementary metrics for locating linguistic information across BERT layers: scalar mixing weights estimate which layers are most useful, while cumulative scoring estimates when predictions become correct.
- Scalar Mixing Weights: Scalar mixing weights identify which layers, in combination, a probing classifier finds most relevant for each task.The learned weights are trained jointly with the probing classifier and interpreted as evidence of task-related information in corresponding layers.
- Scalar Mixing Weights: The mixing-weight center of gravity summarizes the average layer attended to for each task.Higher values indicate that task-relevant information is captured by higher layers.
- Cumulative Scoring: Cumulative scoring measures how much performance improves when each additional encoder layer becomes available.It uses classifiers that attend to a layer and all preceding layers, comparing the resulting F1 scores incrementally.
- Cumulative Scoring: Expected layer summarizes the layer at which probing predictions become correct using differential scores, while excluding layer-0 examples and remaining headroom.This is a pseudo-expectation focused on contextual encoder layers.
- Cumulative Scoring: Adding layers generally increases F1, but distracting features can cause overfitting and performance drops in the last 1–2 layers.The paper notes this behavior particularly in Figure 2.
4 Results
Across BERT-large, the probing metrics reveal an ordered progression from syntactic to semantic information, while different metrics expose complementary aspects of when tasks become solvable and where information is localized.
- Linguistic Patterns: POS, constituents, dependencies, semantic roles, and coreference appear in that order across the network.Basic syntactic information appears earlier, while higher-level semantic information appears in later layers.
- Linguistic Patterns: Syntactic information concentrates on a few layers, whereas semantic information is generally distributed across the network.Relation and semantic proto-role weights are close to uniform, with nontrivial examples resolved gradually across nearly all layers.
- Comparison of Metrics: Differential scores peak in layers 1-7, while learned mixing weights concentrate in layers 9-20 of BERT-large.Many examples are classified early through heuristic shortcuts, but the highest learned weights tend to occur at or after the last layer that improves F1.
- Comparison of Metrics: Continued improvement through the highest layers and diffuse mixing weights characterize semantic relations and semantic proto-roles.For entity types, higher-layer improvements may reflect fine-grained distinctions such as Organization versus Geopolitical Entity, despite many examples being resolved with limited context.
- Comparison of Encoders: The same task ordering appears in 12-layer BERT-base, with representations concentrated at similar relative positions near the model top.This pattern is described as a stretching effect across BERT-base and BERT-large.
- Per-Example Analysis: Per-example decisions can deviate from the average pipeline order, with later semantic information revising earlier entity or tagging decisions.Examples include Toronto changing from GPE to ORG after semantic-role resolution and china today changing to a proper noun before entity and role updates.
5 Conclusion
The paper probes how BERT layers resolve syntactic and semantic structure using two complementary measurements. It finds an aggregate pipeline order but also out-of-order, example-specific revisions driven by higher-level information.
- Conclusion: Edge probing uses scalar mixing weights and cumulative scoring to examine how BERT layers resolve sentence structure.Mixing weights are learned from training data, whereas cumulative scoring is measured on an evaluation set.
- Conclusion: BERT exhibits a consistent aggregate ordering while sometimes resolving individual examples out of order.Predicate-argument relations can help disambiguate lower-level decisions such as part-of-speech.
- Conclusion: The findings provide evidence that deep language models represent hierarchical syntactic and semantic abstractions and model interactions between them.
A.1 Comparison of Encoders
Comparisons across BERT variants show that task representations preserve their relative ordering and occupy similar relative positions, while ELMo emphasizes its first LSTM layer.
- Comparison of Encoders: The most important layers occupy roughly the same relative positions in 24-layer BERT-large and 12-layer BERT-base.Tasks generally appear in the same order in both models.
- Comparison of Encoders: In ELMo, the first LSTM layer is most informative across all tasks.This corroborates earlier observations, while relation and SPR weights remain only weakly concentrated.
A.2 Additional Examples
The additional examples illustrate how BERT’s layer-wise representations support sequential linguistic decisions, with higher-level interpretations revising earlier hypotheses. The figures trace both task-level layer patterns and token-level changes across POS, constituency, entity, and coreference annotations.
- Token-level traces: The “basque” example shows a proper-noun decision in layer 2 preceding a revision from an ordinary noun phrase to a nominal mention in layers 3–4.From layer 3 onward, the model considers “petro basque” either an organization or a national or religious group without strongly disambiguating them.
- Token-level traces: The second “today” example begins with date and common-noun hypotheses, then identifies the TV show by layer 4 and revises constituent and part-of-speech predictions.The trace demonstrates that later interpretations can alter earlier lower-level analyses.
- Token-level traces: Coreference processing can resolve some edges immediately, while other remaining non-coreferent edges are resolved in several later stages.One trace resolves the semantic role of “to help him” in layers 5–7 before resolving the non-coreference of “him” and “he” at layer 8.