Source-linked AI summary
Designing and Interpreting Probes with Control Tasks
John Hewitt, Percy Liang
TL;DR
The paper asks whether probe accuracy reflects linguistic structure in representations or the probe’s own learned task. It introduces random word-type control tasks and selectivity, finding that common probes can be insufficiently selective and that ELMo2 may better support selective part-of-speech probing despite slightly lower accuracy.
Problem
High probe accuracy leaves unresolved whether representations encode linguistic structure or whether probes have learned the task themselves.
Method
The paper constructs random word-type control tasks for part-of-speech tagging and dependency edge prediction, and evaluates probe families, layers, hyperparameters, and regularization using selectivity.
Results
Popular ELMo probes are not selective; linear and bilinear probes are more selective at similar accuracy, dropout is ineffective for MLP selectivity, and ELMo2 is more selective than ELMo1.
Takeaways & Limitations
Selectivity provides context for interpreting probing accuracy and for comparing representation layers, revealing distinctions that linguistic-task accuracy alone can miss.
Takeaways & Limitations
Dependency edge prediction evaluation does not constrain predictions to form a tree and omits the sentence-external ROOT token.
Abstract
from arXiv · showhide
Probes, supervised models trained to predict properties (like parts-of-speech) from representations (like ELMo), have achieved high accuracy on a range of linguistic tasks. But does this mean that the representations encode linguistic structure or just that the probe has learned the linguistic task? In this paper, we propose control tasks, which associate word types with random outputs, to complement linguistic tasks. By construction, these tasks can only be learned by the probe itself. So a good probe, (one that reflects the representation), should be selective, achieving high linguistic task accuracy and low control task accuracy. The selectivity of a probe puts linguistic task accuracy in context with the probe's capacity to memorize from word types. We construct control tasks for English part-of-speech tagging and dependency edge prediction, and show that popular probes on ELMo representations are not selective. We also find that dropout, commonly used to control probe complexity, is ineffective for improving selectivity of MLPs, but that other forms of regularization are effective. Finally, we find that while probes on the first layer of ELMo yield slightly better part-of-speech tagging accuracy than the second, probes on the second layer are substantially more selective, which raises the question of which layer better represents parts-of-speech.
1 Introduction
The paper questions whether high probe accuracy reflects linguistic structure in representations or the probe’s own ability to learn and memorize tasks. It introduces control tasks and selectivity to interpret probing results more carefully.
- Motivation: High linguistic-task accuracy does not establish that a representation encodes linguistic structure, because expressive probes can learn tasks from lossless representations.This motivates evaluating probe behavior beyond standard accuracy.
- Control tasks: Control tasks assign random outputs to word types while preserving the linguistic task’s input and output spaces.They can only be learned through probe memorization of the word-type mapping.
- Selectivity: Selectivity is the difference between linguistic-task accuracy and control-task accuracy, favoring probes with high linguistic accuracy and low control accuracy.It places accuracy in the context of a probe’s capacity to make decisions independently of linguistic properties.
- Findings: 97.3 accuracy versus 92.8 control task accuracy yields only 4.5 selectivity for an MLP part-of-speech probe.A linear probe reaches 97.2 accuracy, 71.2 control task accuracy, and 26.0 selectivity, indicating higher selectivity at similar accuracy.
- Findings: Dropout does not consistently improve MLP selectivity, whereas 10-dimensional hidden states, smaller training samples, and weight decay can improve selectivity at similar linguistic accuracy.The results indicate that conventional complexity control may not align with selectivity.
- Findings: ELMo1 probes achieve 97.2 part-of-speech accuracy versus 96.6 for ELMo2, but ELMo2 achieves 31.4 selectivity versus 26.0 for ELMo1.The accuracy gap is 0.6 in favor of ELMo1, while the selectivity gain is 5.4 in favor of ELMo2; word identity may be more accessible in ELMo1.
2 Control Tasks
Control tasks assign deterministic, independently random outputs to word types while preserving each linguistic task’s output space. Their accuracy measures how much a probe must memorize word identities rather than exploit linguistic information in the representation.
- Construction: Control tasks map each word token’s type through a randomly sampled, deterministic behavior to produce an output in the corresponding linguistic task’s output space.The behavior is sampled independently for each vocabulary item and then applied consistently whenever that word type appears.
- Part-of-speech tagging: For part-of-speech tagging, each word type is assigned one of 45 possible tag outputs, sampled using the empirical token distribution of the linguistic task.This preserves the tagset and approximately matches the marginal label distribution while removing linguistic structure from the word-type assignments.
- Dependency edge prediction: For dependency edge prediction, each word type is uniformly assigned one of three behaviors: attach to itself, the first token, or the last token.These behaviors provide length-independent outputs that share the dependency task’s output space.
- Dependency edge prediction: Unlike dependency parsing, dependency edge predictions are evaluated independently rather than required to form a tree, and the sentence root is omitted.The control and linguistic tasks therefore share output spaces despite differing in structural constraints and root handling.
- Interpretation: Control-task performance is bounded by training-set word-type coverage and requires the probe to identify each representation’s corresponding word type before retrieving its assigned output.Thus, high control accuracy indicates memorization capacity rather than evidence that the representation encodes the linguistic property.
3 Experiments on Probe Selectivity
The experiments compare probe families and complexity controls using linguistic-task accuracy alongside selectivity. They show that default MLPs often trade modest accuracy gains for poor selectivity, whereas simpler probes and targeted constraints can preserve accuracy while improving selectivity.
- Experimental setup: The study compares linear, bilinear, and MLP probes across hyperparameters controlling rank, hidden-state size, and regularization.It evaluates whether probe families can achieve both high linguistic-task accuracy and high selectivity on ELMo representations.
- Probe families: Linear probes match MLP part-of-speech accuracy at 97.2 versus 97.3 while achieving 26.0 versus 4.50 selectivity.For dependency edge prediction, bilinear probes reach 16.7 selectivity versus -0.7 for MLP-1 despite lower accuracy.
- Complexity control: Dropout does not consistently improve MLP selectivity, with one example decreasing from 4.2 to 3.4 as dropout is applied.Across tested settings, dropout and early stopping were the only complexity controls that did not reliably produce high accuracy and high selectivity.
- Complexity control: Ten-dimensional MLP hidden states increase selectivity while maintaining high linguistic-task accuracy for part-of-speech tagging, indicating that common hundreds-dimensional MLPs are overparameterized.Training-example limits help part-of-speech tagging but substantially reduce dependency accuracy, while suitable weight decay can improve selectivity, especially for dependency prediction.
- Discussion: The most selective tested probes are linear or bilinear, but MLPs remain most accurate on dependency edge prediction even after selectivity-oriented tuning.This pattern suggests that some syntactic-tree information is not available to a bilinear function, whereas much part-of-speech information is linearly extractable.
- Error analysis: MLPs improve distinctions such as noun–adjective phrases but more often mislabel proper-noun tags as NNPS, while linear probes are less prone to that error.The authors hypothesize that greater MLP expressivity enables combining spurious plurality and proper-noun cues.
4 Selectivity Differences Confound Layer Comparisons
Layer comparisons based only on linguistic accuracy can be confounded by probe memorization. ELMo2 probes are more selective than ELMo1 probes despite slightly lower part-of-speech accuracy.
- Probes on ELMo2 are more selective than those on ELMo1 across probe types, part-of-speech tagging, and dependency head prediction.
- 31.4 selectivity on ELMo2 versus 26.0 on ELMo1 yields a gain of 5.4 for the linear part-of-speech probe.
- 96.6 ELMo2 linguistic task accuracy versus 97.2 ELMo1 accuracy represents a loss of 0.6 for the same linear probe.
- The results offer the alternative hypothesis that probes use word identity to predict part-of-speech, with that feature less available in ELMo2.
- ELMo2 and Proj0 achieve similar part-of-speech accuracy, but ELMo2 is more selective and therefore relies less on word identity features.
5 Related Work
Related work established probing as a way to inspect linguistic properties in representations, while this paper complements control representations with random type-level control tasks. Prior studies also linked probe accuracy to memorization and training-set size.
- Probing has extracted sentence, word, and word-pair properties using vectors or vector pairs from distributional and contextual representations.
- Control representations such as non-contextual embeddings and random-weight models complement this paper’s control tasks, which measure decoding of random functions from the target representation.
- Zhang and Bowman found that untrained and trained contextualizers had nearly identical part-of-speech accuracy, while reducing training data revealed an advantage for trained models.
- Their memorization analysis found word identities easiest to recover from untrained contextualizers’ representations.
- Random type-level control tasks differ from prior completely random example-level tasks by retaining strong non-linguistic structure.
6 Conclusion
The paper frames probing as an interaction between representations and probe capacity, and proposes selectivity to interpret that interaction. Its results favor simpler or regularized probes and show that selectivity changes conclusions about layer comparisons.
- Probes should be viewed as craftspeople whose performance depends jointly on the representation and the probe, rather than on the representation alone.
- Control tasks are learnable only by the probe, so selectivity measures whether high linguistic accuracy depends on linguistic properties of the representation.
- Linear and bilinear probes achieve higher selectivity at similar part-of-speech accuracy to MLP probes.
- MLP probes can be redesigned for higher selectivity with relatively small dependency-edge accuracy costs, but often not through dropout.
- Marginal ELMo1–ELMo2 part-of-speech accuracy differences correspond to large selectivity differences, while ELMo2 matches a random baseline with much higher selectivity.
- The authors hope control tasks and selectivity will improve interpretation of probing results and understanding of learned representations.