Source-linked AI summary
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong Wu, Yun Chen, Ben Kao, Qun Liu
TL;DR
Supervised probes can confound a pre-trained model’s linguistic knowledge with knowledge learned by the probe’s added parameters. The paper introduces parameter-free Perturbed Masking to recover inter-word structure from BERT, finding useful syntactic patterns and downstream dependency structures that match or exceed a human-designed schema.
Problem
Added probe parameters make it unclear whether linguistic information comes from the pre-trained model or the probe itself.
Method
Perturbed Masking uses two-stage masking within BERT’s masked language-modeling objective to measure inter-word impact without additional probing parameters.
Results
Induced dependency structures perform on-par with or better than a parser-provided, linguist-designed dependency schema in a downstream task.
Takeaways & Limitations
The impact matrices and induced trees support the conclusion that BERT encodes rich syntactic properties and model document structure.
Takeaways & Limitations
The parameter-free probe is not guaranteed to recover human-designed syntax and instead reveals the natural syntax acquired by BERT from self-supervised learning.
Abstract
from arXiv · showhide
By introducing a small set of additional parameters, a probe learns to solve specific linguistic tasks (e.g., dependency parsing) in a supervised manner using feature representations (e.g., contextualized embeddings). The effectiveness of such probing tasks is taken as evidence that the pre-trained model encodes linguistic knowledge. However, this approach of evaluating a language model is undermined by the uncertainty of the amount of knowledge that is learned by the probe itself. Complementary to those works, we propose a parameter-free probing technique for analyzing pre-trained language models (e.g., BERT). Our method does not require direct supervision from the probing tasks, nor do we introduce additional parameters to the probing process. Our experiments on BERT show that syntactic trees recovered from BERT using our method are significantly better than linguistically-uninformed baselines. We further feed the empirically induced dependency structures into a downstream sentiment classification task and find its improvement compatible with or even superior to a human-designed dependency schema.
1 Introduction
Probing classifiers evaluate linguistic information in pre-trained representations, but their added parameters make it unclear how much knowledge comes from the model versus the probe. Perturbed Masking addresses this concern with a parameter-free analysis that extracts inter-word information and evaluates induced structures in linguistic and downstream tasks.
- Motivation: Probing classifiers use pre-trained representations to perform supervised linguistic tasks, treating probe performance as evidence that the model encodes linguistic properties.A probe is a simple neural network with additional parameters trained on the task.
- Motivation: Additional probe parameters make results difficult to interpret because the probe may learn the downstream task and encode the linguistic information itself.This leaves uncertain whether the pre-trained model or probe captures the relevant information.
- Approach: Perturbed Masking is a parameter-free technique that measures one word’s impact on predicting another through the masked language-modeling objective.The resulting inter-word information is used to induce global linguistic properties such as dependency trees.
- Evaluation: The paper evaluates the technique across linguistically driven tasks, including syntactic parsing and discourse dependency parsing.These evaluations are intended to complement and quantitatively assess claims from recent probing work.
- Evaluation: Empirically induced dependency structures perform on-par with or better than a parser-provided, linguist-designed dependency schema in a downstream task.The authors present this comparison as an insight into BERT’s downstream success.
2 Perturbed Masking
Perturbed Masking estimates inter-word impact by comparing BERT representations after progressively masking a target and context token. Repeating this procedure yields token- or span-level impact structures that support linguistic analysis and tree induction.
- Token Perturbation: Perturbed Masking measures the impact of one word on another word’s prediction in BERT’s masked language-modeling setting.The derived inter-word information supports later analysis.
- Token Perturbation: For tokens xi and xj, the method first masks xi, then additionally masks xj, and compares the resulting representation at xi.This two-stage perturbation isolates the effect of the context token on the target representation.
- Token Perturbation: The distance function d compares the two representations, with experiments using Euclidean distance or a probability-based difference.The probability-based option compares the predicted probability of token xi under the two representations.
- Impact Matrices: Repeating perturbations over token pairs produces an impact matrix F, from which syntactic trees can be extracted and compared with benchmark ground truth.For split words, all subword tokens are masked together to obtain word-level impact matrices.
- Span Perturbation: Span-level perturbation masks all tokens in a span simultaneously and averages their token representations to analyze relations between phrases, clauses, or paragraphs.The document is represented as non-overlapping spans, enabling preliminary analysis of document structure.
3 Visualization with Impact Maps
Impact maps visualize token-to-token influence as heatmaps, revealing patterns that suggest syntactic structure and motivate quantitative evaluation.
- An impact map is a heatmap visualization of a sentence’s impact matrix.
- Impact matrices were extracted from BERT using 1,000 sentences from the English PUD treebank.The setup followed BERT’s pre-training preprocessing, and an example impact map was shown in Figure 1.
- Vertical and horizontal stripes indicate that some words strongly affect predictions for other words, including preceding tokens.The word “different” provides an example of a strong vertical stripe above the main diagonal.
- Dependency: Darker links between syntactically closer words, such as “media” and “transitions,” suggest that the matrices contain dependency-relevant information.The observed contrast is stronger than the comparable impacts expected from a syntactically uninformed model.
- Other Structures: Grouped contiguous chunks and strong inter-chunk impacts suggest phrase-level composition, including a larger verb phrase.The observations are presented as evidence that BERT may capture language compositionality.
- These visual observations motivate extracting dependency trees from impact matrices and evaluating them quantitatively.
4 Syntactic Probe
The syntactic probes recover dependency and constituency structures from BERT using impact matrices and graph- or split-based parsing, without supervised syntactic training. BERT outperforms several uninformed or specialized baselines on selected measures, while dependency gains remain largely local.
- Dependency Probe: Perturbed Masking extracts an impact matrix, which graph-based algorithms use to induce unlabeled dependency trees for comparison with linguistic annotations.The evaluation uses Eisner for projective trees and Chu-Liu/Edmonds for non-projective trees.
- Dependency Probe: +41.7 UAS on WSJ10-U and +31.5 UAS on PUD are the best improvements of Eisner+Dist over the random BERT baseline.The Dist distance method also significantly outperforms Prob on both evaluated datasets.
- Dependency Probe: The induced dependency structures do not go much beyond the strong right-chain baseline, suggesting that the recovered relations are mostly simple and local.UAS is also sensitive to annotation variation, motivating the use of UUAS and NED alongside it.
- Dependency Probe: +12.2 UUAS and +28.4 NED over the right-chain baseline indicate inter-word dependencies despite disagreement with one human-designed governor-dependent schema.Some discrepancies are syntactically valid, such as choosing “on” rather than “Sunday” as the head of “on Sunday.”
- Constituency Probe: MART recursively splits sentences at positions maximizing within-constituent impact and minimizing cross-constituent impact.Unlike ON-LSTM and PRPN, MART derives splits directly from the impact matrix rather than an explicitly modeled syntactic-distance vector.
- Constituency Probe: BERT outperforms most constituency baselines on PTB23, except the second layer of ON-LSTM, despite those baselines having architectures designed for unsupervised parsing.BERT’s constituent knowledge emerges from self-supervised training on unlabeled text.
- Constituency Probe: Without the introduced right-branching bias, MART loses only 0.9 F1 on PTB23, indicating robustness to that comparison adjustment.The bias was introduced to ensure a fair comparison with systems whose parsing algorithms favor right-branching English trees.
- Constituency Probe: BERT achieves 51.9% accuracy on SBAR, about 3.4 times ON-LSTM’s accuracy, while remaining competitive on VP and PP tags.The paper offers long contiguous pretraining sequences and random token masking as possible interpretations of the clause-level result.
5 Discourse Probe
The discourse probe tests whether BERT captures document-level structure by analyzing relations among elementary discourse units. On SciDTB, both tested parsers substantially outperform a random BERT baseline, although they trail a strong left-chain prior.
- Discourse Probe: The probe represents documents as coherent elementary discourse units connected by discourse relations and investigates whether BERT captures this structure.The evaluation uses SciDTB because PDTB omits whole-document structure and RST-DT lacks non-projective structures while introducing intermediate nodes.
- Results: +28 UAS over the random BERT baseline is achieved by both Eisner and CLE on the discourse probing task.The result is reported in Table 4’s discourse-parser comparison.
- Results: Adjacent-EDU discourse relations receive decent accuracy, perhaps reflecting BERT’s next sentence prediction pretraining task.The proposed explanation is explicitly presented as tentative.
- Results: Both discourse probes fall behind the left-chain baseline, which benefits from a strong prior that principal clauses usually precede subordinate clauses.The authors connect the finding to BERT’s success on downstream tasks whose inputs contain paragraphs, such as question answering.
6 BERT-based Trees VS Parser-provided Trees
The paper compares dependency trees induced from BERT with parser-provided and trivial trees by measuring their utility for aspect-based sentiment classification. BERT-induced trees are on-par with SpaCy on LAPTOP and marginally better on RESTAURANT, while Eisner trees outperform chain baselines.
- Interpretation: BERT’s induced structures can disagree substantially with linguistically motivated trees while remaining empirically useful for downstream tasks.The paper reports this broader pattern for ABSC and related downstream evaluation.
- Method: The study replaces ground-truth dependency trees in structure-aware networks with trees induced from BERT and estimates tree effectiveness from downstream improvements.The model uses dependency distances to weight context words relative to an aspect.
- Experimental setup: ABSC evaluates whether dependency trees induced from BERT improve downstream aspect-based sentiment classification.The experiments use LAPTOP and RESTAURANT reviews with Accuracy and Macro-Averaged F1.
- Baselines: SpaCy trees provide a human-designed dependency reference, while left-chain and right-chain trees serve as trivial baselines.SpaCy reports an UAS of 94.5 on English PTB.
- Results: BERT-induced trees are on-par with SpaCy on LAPTOP and marginally better on RESTAURANT for downstream performance.LAPTOP is more difficult because its sentences are generally longer, making dependency induction more challenging.
- Results: Eisner trees consistently outperform the right-chain and left-chain baselines.The comparison supports investigating how encoded structure affects downstream performance.
7 Related Work
Related work probes linguistic structure in pre-trained models through supervised classifiers, attention analysis, controlled outputs, and input perturbations. These approaches motivate parameter-free analysis because supervised probes may conflate information in the representation with information learned by the probe.
- Supervised probing: Probing classifiers evaluate linguistic properties ranging from sentence length and tree depth to dependency and constituency parsing.These probes use model representations or attention-related features to perform supervised tasks.
- Supervised probing: High probe accuracy does not distinguish whether syntactic information is encoded by the pre-trained representation or learned by the probe itself.The probe introduces additional parameters that can encode the supervised task.
- Attention analysis: Attention-based studies extract syntactic trees from individual attention heads and layers to compare their tracking of syntactic relations against baselines.Prior work includes dependency-tree extraction from self-attention weights.
- Behavioral tests: Controlled-output studies test syntax-sensitive behaviors such as long-distance subject–verb agreement in language-model predictions.These tests show BERT captures agreement patterns well in general but do not assess complex dependency or constituency structures.
- Perturbation methods: Earlier perturbation methods alter input sequences once, whereas Perturbed Masking uses a two-step masking strategy to estimate inter-word effects.The prior methods target reducibility or word alignment rather than the paper’s broader structural analysis.
8 Discussion & Conclusion
The paper presents Perturbed Masking as a parameter-free complement to supervised probing for interpreting BERT. Its impact matrices yield syntactic and document-structure analyses, and the induced dependency structures can improve downstream accuracy despite differing from human-designed syntax.
- Discussion: Perturbed Masking complements existing BERT probing methods by analyzing linguistic structure without adding probe parameters.The paper frames the method as complementary rather than as a replacement for supervised probes.
- Interpretation: BERT encodes rich syntactic properties, although its induced syntax can differ substantially from linguist-designed structures.The authors describe their probe as providing a lower bound on BERT’s unsupervised syntactic parsing ability.
- Method: A two-stage perturbation procedure produces impact matrices that capture inter-word correlations from BERT outputs.The matrices mirror the function of attention mechanisms while emerging from model outputs rather than intermediate representations.
- Scope: The method extracts syntactic trees and extends analysis to document structure, including how BERT models long sequences.The paper also proposes future applications to other linguistic properties and downstream systems.
- Downstream relevance: Feeding BERT-induced dependency structures into a downstream system can improve accuracy compatibly with or even beyond a human-designed dependency schema.This finding connects structural disagreement with empirical usefulness in downstream applications.