Source-linked AI summary
Visualizing and Measuring the Geometry of BERT
Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, Martin Wattenberg
TL;DR
The paper asks how BERT internally represents the linguistic features that support its broad task flexibility. It investigates syntax and semantics with visualizations, probes, and mathematical analysis, finding evidence for distinct geometric subspaces and fine-grained word-sense representations.
Problem
BERT’s broad task flexibility raises the question of which generally useful linguistic features it extracts and how those features are represented internally.
Method
The paper studies BERT’s internal geometry using visualizations, attention probes, and mathematical analysis of tree embeddings.
Results
The paper finds evidence that syntactic information is encoded in attention vectors and that semantic information occupies relatively low-dimensional subspaces.
Takeaways & Limitations
The results suggest BERT’s internal geometry may contain separate linear subspaces for different syntactic and semantic information.
Takeaways & Limitations
For p < 2, some trees have no power-p embedding, while for p > 2 every tree has one but lacks a similarly simple geometric description.
Abstract
from arXiv · showhide
Transformer architectures show significant promise for natural language processing. Given that a single pretrained model can be fine-tuned to perform well on many different tasks, these networks appear to extract generally useful linguistic features. A natural question is how such networks represent this information internally. This paper describes qualitative and quantitative investigations of one particularly effective model, BERT. At a high level, linguistic features seem to be represented in separate semantic and syntactic subspaces. We find evidence of a fine-grained geometric representation of word senses. We also present empirical descriptions of syntactic representations in both attention matrices and individual word embeddings, as well as a mathematical argument to explain the geometry of these representations.
1 Introduction
BERT’s broad task flexibility motivates investigating which linguistic features it extracts and how those features are represented internally. This paper extends prior analyses with geometric studies of syntax and semantics.
- Motivation: BERT can be fine-tuned to achieve state-of-the-art performance across a wide variety of NLP applications.Its flexibility suggests that one pretrained model extracts generally useful features from raw text.
- Related work: Prior neural-network studies found layerwise changes, meaningful units, and meaningful directions in internal activation spaces.These investigations contributed to a broader understanding of network behavior.
- Related work: Earlier language-model analyses found rich linguistic representations, including syntactic features in RNNs and BERT and geometric representations of parse trees.The cited work provides context for studying BERT’s internal geometry.
- Contributions: The paper finds grammatical representations in attention matrices and suggests that BERT distinguishes word senses at a fine level.It also gives mathematical arguments for parse-tree embeddings and finds that much semantic information lies in a relatively low-dimensional subspace.
2 Context and related work
BERT converts token sequences into contextual vectors through stacked transformer layers whose attention matrices encode token-to-token interactions. Prior work and the paper’s framing motivate studying how syntactic and semantic information coexist geometrically.
- BERT architecture: BERT outputs one context embedding for each input token, incorporating information about that token’s context.Its input consists of words or word pieces represented as a token sequence.
- BERT architecture: BERT combines wordpiece, position, and segment information before processing the sequence through multiple transformer layers.Each attention head contains a scalar for every ordered pair of tokens.
- Geometric representation: Neural language models must translate discrete sentences into geometric representations in high-dimensional vector spaces.The motivating questions concern which linguistic features are represented and in what form.
- Related work: Prior work found semantic and grammatical distinctions represented by directions in vector spaces and syntactic information recoverable from neural representations.These findings span Word2Vec and RNN-based models.
- Related work: Simple classifiers can extract many NLP-pipeline properties from BERT context embeddings, while visualization-based work suggests attention matrices encode relations between words.The properties include part-of-speech tagging, coreference resolution, and dependency labeling.
- Related work: A structural probe found squared distances between transformed BERT context embeddings roughly proportional to dependency-tree distances.The paper uses this result as motivation for analyzing why squaring distance is necessary.
- Research questions: The paper asks whether other intermediate representations exist and how different kinds of information coexist in a single vector.These questions follow from evidence that language-processing networks represent both semantic and syntactic information.
3 Geometry of syntax
The paper investigates how BERT geometrically represents syntax, combining attention probes, visualizations, and mathematical analysis. It finds syntactic information in attention vectors and characterizes tree embeddings whose squared Euclidean distances reflect dependency structure.
- Attention probes and dependency representations: Model-wide attention vectors are formed from the attention entries for an ordered token pair across every head and layer.The study applies these vectors to dependency-relation classification using parsed Penn Treebank sentences.
- Attention probes and dependency representations: A linear attention probe predicts whether two tokens have a dependency relation with 85.8% accuracy and identifies its type with 71.9% accuracy.The probe uses model-wide attention vectors concatenating attention values across heads and layers.
- Mathematics of embedding trees in Euclidean space: A tree cannot generally be embedded isometrically in Euclidean space, motivating power-p embeddings that relate powered Euclidean distances to tree distance.The paper focuses on the power-2 case, called a Pythagorean embedding.
- Mathematics of embedding trees in Euclidean space: Every tree with n nodes has a Pythagorean embedding in R^(n−1), constructed using orthogonal unit basis vectors along tree branches.The canonical embedding makes neighboring edges unit length and mutually orthogonal.
- Mathematics of embedding trees in Euclidean space: Random unit branch directions approximate Pythagorean embeddings in sufficiently high-dimensional spaces because random vectors are nearly orthogonal.This provides a locally constructible alternative to the canonical construction.
- Visualization of parse tree embeddings: Average squared embedding distances vary by dependency relation, from about 1.2 for compound, prt, and advcl to 2.5 for mwe, parataxis, and auxpass.These systematic differences indicate an additional quantitative aspect beyond traditional dependency grammar.
4 Geometry of word senses
BERT’s context embeddings represent word senses geometrically: different meanings form separated clusters with fine-grained internal structure. Quantitative tests support sense information in a lower-dimensional semantic subspace that complements syntax, while altered context can increase sense errors.
- 4.1 Visualization of word senses: BERT-base context embeddings for 1,000 sentences are visualized with UMAP to explore how context affects word-sense geometry.The data come from English-language Wikipedia introductions, and users can choose the BERT layer supplying each word’s context embedding.
- 4.1 Visualization of word senses: Different word senses generally form spatially separated clusters, often containing additional structure for finer shades of meaning.For “die,” three meanings form crisp clusters, with one cluster showing a scale related to the number of people dying.
- 4.2 Measurement of word sense disambiguation capability: 71.1 F1: a nearest-neighbor classifier using sense centroids exceeds the reported baseline and ELMo score for word-sense disambiguation.Accuracy increases monotonically through BERT’s layers, while a semantic-probe method reaches 71.5.
- 4.2.1 An embedding subspace for word senses?: Earlier-layer embeddings show more substantial probe-based gains than final-layer embeddings, indicating that their geometry contains semantic information not evident from the untransformed representations.The final-layer improvement is described as modest, whereas earlier-layer performance improves more dramatically.
- 4.2.1 An embedding subspace for word senses?: The results support encoding word-sense information in a lower-dimensional subspace complementary to the syntactic subspace.This offers the paper’s proposed resolution to how one vector can encode both syntax and semantics.
- 4.3.2 Results: 8.23% versus 2.43%: final-layer sense misclassification is significantly higher when sentences are concatenated with another sentence using the keyword in an opposing sense.The average similarity ratio also decreases after concatenation, while random-sentence concatenation does not change individual similarity ratios.
5 Conclusion and future work
The paper finds evidence that BERT represents syntax and semantics through structured geometric subspaces, while leaving the decomposition and architectural implications for future work.
- Attention matrices contain syntactic representations, with particular directions encoding dependency relations.
- A mathematical argument justifies the squared-distance tree embedding identified by Hewitt and Manning.
- BERT appears to contain semantic subspaces, and word-sense disambiguation errors may correspond to changes in the geometry of word meaning.
- The authors conjecture that BERT’s internal geometry decomposes into separate linear subspaces for different syntactic and semantic information.
- Identifying additional meaningful subspaces and relating internal geometry to transformer architecture remain important directions for future research.
6 Appendix
The appendix explains why exact tree isometries in Euclidean space are highly restricted, using a four-point tree as a counterexample.
- An exact Euclidean isometry cannot embed a four-point tree with one root and three children.Equal root-to-child distances and pairwise child distances force incompatible collinearity constraints.
- Because the four-point tree is impossible to embed, only chain-shaped trees admit such exact embeddings.
- Isometric embeddings are not the only restricted case; power-p embeddings may also fail when p < 2.
Proof of Theorem 2
The proof shows that sufficiently large star-shaped trees cannot have power-p embeddings for any p < 2.
- For 1 < p < 2, the proof considers a star-shaped tree with one root and k children.
- The root is placed at the origin, sending the remaining vertices to unit vectors whose pairwise power-p distances must equal 2.
- Among k unit vectors, at least one pair has inner product at least −1/(k −1), constraining its Euclidean distance.
- For any p < 2, choosing k sufficiently large makes the resulting power-p distance less than 2, contradicting the required embedding distances.
Random branch embeddings are probably approximately Pythagorean
The paper constructs random branch embeddings as local approximations to Pythagorean tree geometry and compares them with BERT parse-tree embeddings and semantic-syntactic probes.
- Random branch embeddings use random unit-like Gaussian branches, which become nearly orthogonal in sufficiently high-dimensional spaces.This makes the Pythagorean embedding condition approximately hold using local parent-child information.
- For nodes at tree distance m, the random branch embedding’s Euclidean distance has mean m.The theorem also gives a standard deviation, though its displayed value is not included in the passage.
- Canonical Pythagorean embeddings place neighboring edges at unit distances and make them mutually orthogonal, requiring n−1 dimensions for an n-node tree.
- BERT parse-tree embeddings show clear visual similarity to canonical Pythagorean and random branch embeddings in PCA projections, unlike the completely random control.
- The paper also presents additional BERT parse-tree and word-sense visualizations, including context embeddings for different uses of “lie.”
- The singular values of probe products A^T*B and A*B^T fall faster than those of either probe alone, suggesting syntactic and semantic probes are orthogonal.