PaperlayerView exact revision on arXiv

Source-linked AI summary

Information-Theoretic Probing for Linguistic Structure

Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, Ryan Cotterell

arXiv:2004.03061v2cs.CLcs.LG

TL;DR

The paper asks how probing should measure linguistic information in neural representations and formalizes probing as mutual-information estimation. It develops control functions and evaluates BERT across eleven languages, finding limited gains over type-level baselines and recommending attention to ease of extraction.

  • Problem

    Existing probing practice often favors simple probes to distinguish encoded linguistic structure from task learning, but the paper questions this goal and seeks a formal operationalization.

  • Method

    The authors estimate mutual information with supervised probes, compare contextual representations against control functions, and use a deep MLP to obtain tighter bounds.

  • Results

    BERT improves over a type-level baseline by at most 12% more information, and POS estimates exceed the control in fewer than half of the eleven languages.

  • Takeaways & Limitations

    The results suggest word-level POS labeling may be poorly suited for assessing syntax in contextual embeddings, motivating focus on ease of extraction.

  • Takeaways & Limitations

    The control-task design may strip away natural lexical ambiguity and make the comparison relatively trivial, while selectivity can artificially disfavor memorization.

Abstract

from arXiv · show

The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually ``know'' about natural language. Probes are a natural way of assessing this. When probing, a researcher chooses a linguistic task and trains a supervised model to predict annotations in that linguistic task from the network's learned representations. If the probe does well, the researcher may conclude that the representations encode knowledge related to the task. A commonly held belief is that using simpler models as probes is better; the logic is that simpler models will identify linguistic structure, but not learn the task itself. We propose an information-theoretic operationalization of probing as estimating mutual information that contradicts this received wisdom: one should always select the highest performing probe one can, even if it is more complex, since it will result in a tighter estimate, and thus reveal more of the linguistic information inherent in the representation. The experimental portion of our paper focuses on empirically estimating the mutual information between a linguistic property and BERT, comparing these estimates to several baselines. We evaluate on a set of ten typologically diverse languages often underrepresented in NLP research---plus English---totalling eleven languages.

1 Introduction

The paper formalizes probing as estimating mutual information between representations and linguistic properties, challenging the preference for simple probes. Experiments compare BERT with controls across eleven languages.

  • Probing is framed as estimating mutual information between a representation-valued variable and a linguistic-property variable.
  • The authors argue that the highest-performing probe gives the tightest mutual-information estimate, regardless of probe complexity.
  • The proposed view rejects a strict distinction between a probe learning the task and a representation encoding linguistic structure.
  • In experiments, POS mutual-information estimates from BERT exceed a control in only five of eleven typologically diverse languages.

2 Word-Level Syntactic Probes for Contextual Embeddings

The paper defines word-level syntactic probing through random variables for sentences, analyses, and contextual token representations, then estimates mutual information with supervised probes. It argues that better-performing probes provide tighter bounds.

  • Word-level probes predict POS or dependency labels from an individual token embedding, although the target analysis may require sentential context.
  • 2.1 Notation: The framework models sentences as S, target analyses as T, and contextual token representations as R derived from the entire sentence.
  • 2.2 Probing as Mutual Information: Probing is operationalized as mutual information between the target analysis and representation, but the required true distributions are unknown.
  • 2.2 Probing as Mutual Information: A probe qθ provides an upper bound on conditional entropy and therefore a lower bound on mutual information; the bound tightens as qθ approaches the true conditional distribution.
  • The authors therefore prefer the probe that performs best on the target task, treating the simpler-versus-more-complex probe distinction as a false dichotomy.

3 Control Functions

Control functions contextualize mutual-information estimates by comparing representations with controlled baselines, especially type-level word representations. The gain measures additional information from contextual representations, but its practical estimation has limitations.

  • 3 Control Functions: Control functions c(·) place I(T; R) relative to baseline information encoded by a transformation of the representation.
  • 3.1 Type-Level Control Functions: Type-level controls decontextualize embeddings, allowing the analysis to ask how much contextual information improves target-task performance.
  • 3.1 Type-Level Control Functions: Under the uniqueness assumption, a function can recover word identity from each contextualized embedding, supporting comparison with noncontextualized word embeddings.
  • 3.2 How Much Information Did We Gain?: The gain G measures how much more task information contextualized embeddings contain than a controlled baseline and equals a conditional mutual information.
  • 3.3 Approximating the Gain: The gain is approximated with variational upper and lower bounds optimized using separate probe distributions and empirical cross-entropies.
  • 3.3 Approximating the Gain: The finite-sample approximation can be negative even though the true gain is non-negative, and no remedy has yet been devised.

4 Understanding Probing Information-Theoretically

The paper argues that contextualized embeddings contain no more information about word-level syntactic tasks than the original sentence, while control functions illuminate extractability rather than representation content.

  • 4.1 You Know Nothing, BERT: BERT(S) cannot provide more information about a word-level syntactic task than the original sentence S itself.This follows from treating BERT as a bijection and applying the data-processing inequality.
  • 4.1 You Know Nothing, BERT: The equality I(T; S) = I(T; BERT(S)) is not specific to BERT but follows because the data-processing inequality is tight for bijections.
  • 4.1 You Know Nothing, BERT: Under the paper’s operationalization, searching contextualized embeddings for syntax cannot reveal more syntax than the original sentence already contains.The theorem makes the relevant information available a priori rather than through probing.
  • 4.2 Control Functions: Control functions do not provide information about the representations themselves because contextual representations can be mapped back to their source sentences.For a type-level control function c, the paper gives I(T; W) ≥ I(T; c(R)).
  • 4.3 Discussion: Ease of Extraction: The paper suggests interpreting better representations as those whose encoded information is more accessible through easily learnable probes.Approximating the relevant inverse and control functions may require unreasonable datasets, including all vocabulary words for a random-embedding control.

5 A Critique of Control Tasks

The paper critiques control tasks based on deterministic word-type labels and random outputs, arguing that they can trivialize the task, penalize memorization, and make selectivity a misleading probe-selection criterion.

  • 5.1 Structure and Randomness: Control tasks with deterministic word-type labels remove contextual ambiguity; for example, love can be only a NOUN or only a VERB, never both.The paper argues that combining this structure with random labels may produce a task unlike natural language.
  • 5.1 Structure and Randomness: A zero-parameter retrieval probe can solve the control task by memorizing training labels and guessing the most frequent tag for unseen words.Its accuracy is 1 minus the out-of-vocabulary rate times the test-set error rate of the most-frequent-tag classifier.
  • 5.2 What’s Wrong with Memorization?: Maximizing selectivity favors models that are bad at memorization, although memorization is part of linguistic competence alongside generalization.
  • 5.3 What Low-Selectivity Means: For dependency edge prediction, Hewitt and Liang report that an MLP probe is more accurate and preferable despite its lower selectivity.
  • 5.3 What Low-Selectivity Means: The authors interpret reported failures of less-selective probes as evidence that simple probes can misrepresent learned information, rather than evidence that selectivity is useful.They specifically discuss suffix -s being mislabeled as NNPS and ELMo’s second layer failing to outperform its word-type representations with MLP-1.

6 Experiments

The experiments estimate how much syntactic information multilingual BERT representations provide beyond type-level controls across eleven languages. They use powerful probes for POS and dependency labeling, finding modest contextual gains, especially for POS.

  • Experimental setup: Eleven typologically diverse languages are evaluated with multilingual BERT, fastText, and one-hot controls on POS and dependency labeling.BERT representations average word-piece outputs for each treebank-tokenized word; dependency probes concatenate token and head embeddings.
  • Experimental setup: The study uses deep MLP probes and selects the best validation cross-entropy among 50 trained models to estimate task entropy.Hyperparameters include layers, hidden size, one-hot embedding size, and dropout, selected by random search.
  • POS results: Type-level embeddings capture most POS uncertainty in every analyzed language, while BERT adds only small information gains and sometimes negative gains.The authors attribute apparent negative gains to finite-sample difficulty approximating the control functions.
  • Dependency-label results: BERT improves over type-level embeddings for dependency labeling in all languages, but its estimate exceeds fastText by at most 12% in English.Five lower-resource languages show gains below 5%.
  • Interpretation: Overall, multilingual BERT appears to encode little more syntactic information than simple baselines, even for the more context-dependent dependency task.The results motivate formally defining ease of extraction rather than relying only on probe complexity or information.

7 Conclusion

The paper formalizes probing as estimating conditional mutual information and uses control functions to measure information beyond trivial knowledge. Its multilingual BERT evaluation finds substantial syntax information but only modest gains over type-level baselines, motivating ease of extraction as a future focus.

  • Contributions: The paper defines probing as estimating conditional mutual information and introduces control functions for contextualizing those estimates.The controls measure how much more informative contextual representations are than knowledge judged trivial.
  • Contributions: With perfect probes, probing can reveal information about the language but cannot distinguish what is encoded in the representations themselves.This follows from the paper’s information-theoretic analysis of the operationalization.
  • Future direction: The paper recommends shifting attention from probe size or information to rigorously formalized ease of extraction.It notes that simple probes may still illuminate whether encoded information is structured for downstream use.
  • Empirical conclusion: Across eleven typologically diverse languages, BERT encodes over 76% of POS information and 65% of dependency-label information.These are reported as amounts of information about the tasks in all languages.
  • Empirical conclusion: BERT encodes at most 12% more information than a type-level baseline, and its POS estimate exceeds the control in fewer than half the languages.The authors conclude that word-level POS labeling may not be ideal for studying syntax in contextual embeddings.

A Variational Bounds

This section bounds the estimation error between the probe-based gain and the true gain using two distinct Kullback–Leibler divergences.

  • Variational bounds: The estimation error between the estimated gain Gqθ(T, R, e) and the true gain is bounded above and below by two distinct Kullback–Leibler divergences.The proof first expresses the error as a difference between two KL divergences, then derives bounds using their nonnegativity.

B Further Results

The appendix reports model accuracies for POS and dependency labeling, then examines task ambiguity and derives terms used in the estimation-error analysis.

  • Model accuracies: BERT and fastText achieve high POS-labeling accuracy in all languages except Tamil and Marathi, while one-hot and random embeddings perform considerably worse.The weaker controls generally cannot do better than guessing for word types unseen during training.
  • Model accuracies: Dependency-labeling accuracies follow the same model pattern as POS labeling but are considerably lower overall.
  • Task ambiguity: The tables estimate linguistic-task ambiguity given word types using conditional entropy H(T | id(R)).
  • Task ambiguity: The plug-in entropy estimates should not be treated as accurate approximations because undersampled regimes are known to cause entropy underestimation.
  • Task ambiguity: Most analyzed languages are relatively unambiguous for POS labeling, but uncertainty varies substantially across languages for both POS and dependency labeling.
Loading 2004.03061v2…