Source-linked AI summary
Semantic Structure and Interpretability of Word Embeddings
Lutfi Kerem Senel, Ihsan Utlu, Veysel Yucesoy, Aykut Koc, Tolga Cukur
TL;DR
Word embeddings often have dimensions whose semantic meanings are difficult to interpret, and task performance alone does not show what semantic structure they capture. The paper introduces SEMCAT and a statistical approach to analyze that structure and measure interpretability without human evaluation. Its analysis uses a distributional-distance measure under a normality assumption.
Problem
Embedding dimensions are often difficult to interpret, and stronger benchmark performance does not necessarily reveal the semantic structure they capture.
Source: §I · p. 1Method
The authors introduce SEMCAT, a dataset of words grouped into semantic categories, and use category-word distribution statistics to analyze embedding dimensions and quantify interpretability.
Source: §I · p. 2, A. Dataset · p. 3, B. Semantic Decomposition · p. 4Results
The authors report using qualitative and quantitative tests to substantiate their semantic decomposition and propose a SEMCAT-based interpretability measure that requires no human evaluation.
Source: §I · p. 2, §V · p. 10Takeaways & Limitations
The SEMCAT-based framework can compare embedding methods by the semantic category weights of their dimensions.
Source: §V · p. 10The category-weight metric assumes normally distributed embedding dimensions; the authors suggest nonparametric alternatives when distributions substantially depart from normality.
Source: §V · p. 10
Abstract
from arXiv · showhide
Dense word embeddings, which encode semantic meanings of words to low dimensional vector spaces have become very popular in natural language processing (NLP) research due to their state-of-the-art performances in many NLP tasks. Word embeddings are substantially successful in capturing semantic relations among words, so a meaningful semantic structure must be present in the respective vector spaces. However, in many cases, this semantic structure is broadly and heterogeneously distributed across the embedding dimensions, which makes interpretation a big challenge. In this study, we propose a statistical method to uncover the latent semantic structure in the dense word embeddings. To perform our analysis we introduce a new dataset (SEMCAT) that contains more than 6500 words semantically grouped under 110 categories. We further propose a method to quantify the interpretability of the word embeddings; the proposed method is a practical alternative to the classical word intrusion test that requires human intervention.
I. INTRODUCTION
Word embeddings perform well across NLP tasks, but their individual dimensions are often difficult to interpret, and benchmark performance alone does not reveal the semantic structure they capture. The paper investigates that structure with human-designed categories and proposes an embedding-based interpretability measure that does not require human evaluation.
I. INTRODUCTION
- Individual embedding dimensions are commonly considered uninterpretable, despite embeddings’ success across NLP tasks.The paper notes that dimensions may function as a black box, even though interpretable dimensions could suit tasks requiring semantic interpretation.Source: §I · p. 1
- Benchmark gains do not necessarily reveal an embedding’s semantic structure, motivating systematic assessment of its interpretability.Such assessment could support comparisons among embeddings and improve understanding of the approach.Source: §I · p. 1
- The study uses human-designed categories to investigate latent semantic concepts in embedding dimensions and align interpretations with human judgments.Human interpretation can group words through diverse semantic relations, making categories both numerous and difficult to define objectively.Source: §I · p. 1
- The authors introduce a dataset of more than 6,500 words in 110 categories and propose statistical analysis and a human-free interpretability measure.They report using qualitative and quantitative tests to substantiate their method and contrast the measure with the human-evaluated word intrusion test.Source: §I · p. 2
II. RELATED WORK
Prior interpretability approaches either require human judgments, produce new representations rather than explain dense embedding dimensions, or use metrics that may not reflect human interpretation. The paper’s methods address these gaps with a conceptual category dataset and statistical analysis of word embeddings.
II. RELATED WORK
- Word intrusion evaluations require human judgments for each dimension, while neuroimaging-based JNNSE also requires labor-intensive data collection –.The word intrusion test asks human evaluators to identify an intruder word in a set for each embedding dimension.Source: §II · p. 2
- Sparse-coding approaches learn new, higher-dimensional representations rather than labeling the semantic content of the original dense embedding dimensions [25], [26].The resulting spaces are evaluated on tests such as word similarity and polysemy detection.Source: §II · p. 2
- Embedding-derived interpretability metrics avoid human evaluation, but may not reflect human interpretations [27],.The cited work [27] also reports that orthogonal transformations redistribute, rather than change, total interpretability.Source: §II · p. 2
III. METHODS
- The authors introduce a conceptual category dataset and statistical methods to capture hidden semantic concepts and measure embedding interpretability.The methods are intended to address limitations of the approaches reviewed in the related work.Source: §III · p. 2
A. Dataset
The dataset uses human-defined semantic categories because people group words through varied relations and properties, while existing category resources do not cover that breadth. SEMCAT is built from educational word lists with filtering and a frequent-word vocabulary limit.
A. Dataset
- The authors use human-categorized concepts as a supervised basis for examining which semantic concepts embeddings capture.They argue that clusters formed by unsupervised embeddings may not match human expectations.Source: A. Dataset · pp. 2–3
- Human categories can reflect varied relations and properties, but there is no clear procedure for deciding their scope or membership.People may group words by hypernymy, meronymy, antonymy, synonymy, cross-part-of-speech relations, or shared properties such as color and speed.Source: A. Dataset · p. 3
- SEMCAT is derived from educational word lists by removing nuisance-property groupings, overly specific categories, and words outside the 50,000 most frequent English Wikipedia words.The paper provides summary statistics for SEMCAT and HyperLex and examples of words from six representative SEMCAT categories.Source: A. Dataset · p. 3, Table I · p. 3, Table II · p. 3
B. Semantic Decomposition
The authors measure how strongly each semantic category is represented in each embedding dimension by comparing category-word distributions with those of the remaining vocabulary. They use Bhattacharya distance under a normal-distribution assumption to form category weights.
B. Semantic Decomposition
- The study trains GloVe on the entire English Wikipedia corpus, using a 15-word window, 300-dimensional vectors, minimum occurrence count 20, and the 50,000 most frequent words.It runs GloVe a second time on the same corpus to examine the effects of different initializations of the word vectors prior to training.Source: B. Semantic Decomposition · p. 3
- A category’s mean alone can miss encoding in dense embeddings, where dimensions differ and category words may be tightly grouped despite a mean near the vocabulary mean.The authors illustrate this with jobs-category means across two dimensions and car-category dispersion in another dimension.Source: B. Semantic Decomposition · p. 4
- Category encoding is measured by the distributional distance between category words and all other vocabulary words in a dimension.Bhattacharya distance under a normal-distribution assumption is used to quantify that separation; dimension normality is assessed with a Bonferroni-corrected one-sample Kolmogorov-Smirnov test.Source: B. Semantic Decomposition · p. 4
- The resulting 300 × 110 category-weight matrix assigns a weight to each dimension-category pair, with larger values indicating greater separation from other vocabulary words.Weights range from 0 to infinity; zero corresponds to matching means and variances for the compared distributions.Source: B. Semantic Decomposition · p. 4
C. Interpretable Word Vector Generation
The paper constructs two category-oriented embedding spaces: I uses distribution-based category weights, while I∗ uses category centers. The I mapping standardizes vectors, normalizes weights, and preserves their signed direction.
C. Interpretable Word Vector Generation
- Directly multiplying embeddings by WB is not the proposed mapping; the method first makes their scales compatible with the category weights.The passage says the weights are based on deviations from the general mean and standard deviations.Source: C. Interpretable Word Vector Generation · p. 4
- I maps standardized word vectors using category weights that are normalized across dimensions and signed to preserve each dimension’s positive or negative encoding direction.The procedure standardizes dimensions to zero mean and unit variance, normalizes each category’s total weight to 1, then multiplies by the signed weights.Source: C. Interpretable Word Vector Generation · p. 4
- I∗ is generated by projecting the original word vectors onto category centers, whose weights are the category-word means for each embedding dimension.The paper presents center-based weights as an alternative to the Bhattacharya-distance weights used for I.Source: C. Interpretable Word Vector Generation · pp. 4–5, Figure 1 · p. 5
D. Validation
The validation compares whether I and I∗ organize words by SEMCAT category, including held-out words not used to calculate the weights. It uses sparse category weights and measures the held-out words’ ranks in their corresponding dimensions.
D. Validation
- The validation tests whether I and I∗ represent categories better than category center vectors, using qualitative inspection and quantitative evaluation.The expected structure is that each space’s dimensions correspond to SEMCAT categories and each word’s vector reflects its category membership.Source: D. Validation · p. 5
- Words outside the categories serve as held-out validation data, avoiding evaluation on the same words used to calculate category weights.The passage compares this to using validation rather than training accuracy to assess generalization.Source: D. Validation · p. 5
- For each category, 60% of its words train the weights and the remaining 40% are held out for testing.The random split is repeated for 10 independent selections; the test words’ presence among the top n, 3n, and 5n words in their corresponding dimensions is measured, with n equal to the category’s test-word count.Source: D. Validation · p. 5
- The validation compares sparse I and I∗ spaces built from the largest k weights per category, with k tested at nine values from 5 to 300.The remaining weights are set to zero, and accuracies are weighted across dimensions in proportion to the number of test words in each category.Source: D. Validation · p. 5
E. Measuring Interpretability
The section replaces the human-dependent, binary word intrusion test with an automated, continuous interpretability measure grounded in human-designed semantic categories. It defines category-, dimension-, and embedding-level scores and examines how category representation varies across the embedding.
E. Measuring Interpretability
- Word intrusion requires human judgments and yields a binary decision, while interpretability can vary gradually across dimensions.The section motivates continuous scoring as a more adequate alternative.Source: E. Measuring Interpretability · pp. 5–6
- The 300-by-110 category-weight matrix shows that dimensions differ in how many categories they represent and categories differ in how many dimensions represent them strongly.The figure maps Bhattacharya-distance weights from 0 (black) to 0.63 (white).Source: Figure 2 · p. 6
- The framework checks whether distinctive dimension words co-occur in human-designed categories, using a sufficiently large category set to approximate possible human groupings.The authors note that possible human groupings are theoretically unbounded, so the dataset cannot cover them all.Source: E. Measuring Interpretability · p. 6
- For each dimension–category pair, the method scores category-word overlap among the dimension’s top- or bottom-ranked words, with λ controlling the rank range.λ = 1 is the strictest setting; larger values relax the definition by widening the selected rank range.Source: E. Measuring Interpretability · pp. 6–7
- Metals has the strongest total category representation and pirate the weakest; the authors relate this contrast to word clustering and distribution.Metals contains relatively few, well-clustered words; pirate words are widespread.Source: Figure 3 · p. 6
- Dimension decompositions range from category-focused to broadly distributed, while math is represented by a few dominant dimensions and tools and animals by many.The figure illustrates these patterns with dimensions 2, 6, and 45 and categories math, animal, and tools.Source: Figure 4 · p. 7
- The method takes the stronger directional score for each category, the maximum category score for each dimension, then averages dimension scores for the embedding.This defines category-, dimension-, and overall embedding-level interpretability scores.Source: E. Measuring Interpretability · p. 7
- The evaluation tests GloVe, two semantic spaces, and a random-vector baseline while varying λ from 1 to 10; measured scores may also depend on SEMCAT’s category and word selection.The authors probe dataset sensitivity by subsampling categories and selecting different proportions of words closest to category centers.Source: E. Measuring Interpretability · p. 7
A. Semantic Decomposition
Semantic category representation varies widely across GloVe dimensions and categories, and category-word clustering helps explain those differences. The decompositions show that categories can be concentrated in a few dimensions or distributed broadly across many.
A. Semantic Decomposition
- 255 of 300 GloVe dimensions passed the KS normality test (p > 0.05), while the remaining 45 had a small average test statistic consistent with approximate normality.The average test statistic was 0.0064±0.0016 for the 255 dimensions and 0.0156±0.0168 for the remaining 45.Source: A. Semantic Decomposition · p. 7
- Category weights are broadly distributed across many dimensions of the GloVe embedding, suggesting poor interpretability of the raw space.The examined category-weight distributions are shown in Figure 2.Source: A. Semantic Decomposition · pp. 7–8
- Category representation strength varies substantially, with metals strongest because its relatively few words are well clustered and pirate weakest because its words are widespread.Tightly clustered words yield larger Bhattacharya distances across dimensions; weakly related, widespread words yield smaller distances.Source: A. Semantic Decomposition · p. 8
- Semantic encodings vary by category: math is dominated by a few dimensions, while tools and animals are distributed across many.The 2nd dimension selectively represents sciences, the 45th focuses on housing, rooms, and sciences, and the 6th represents many categories relatively uniformly.Source: A. Semantic Decomposition · p. 8
- The category-retrieval figure compares top-n, 3n, and 5n retrieval, with n varying by category, and reports better performance for Bhattacharya-based weights than category centers.Using k = 25 largest weights from WB performs better than category centers at any k.Source: Figure 7 · p. 9
B. Validation
The proposed category weights distinguish categories better than category-center vectors in retrieval tests. Word decompositions also show that both approaches assign high values to semantically related categories that do not contain the word.
B. Validation
- For window, bus, soldier, and article, semantically related categories that do not contain the word also receive high values in I.Examples for soldier include jobs, people, pirate, and weapons; similar patterns appear for the other words.Source: B. Validation · p. 9
- In I*, word scores are more uniformly distributed across categories than in I, making the category-center approach less discriminative.The compared decompositions are for window, bus, soldier, and article.Source: B. Validation · p. 9
- Using k = 25 largest WB weights yields higher word-retrieval accuracy than category-center weights WC with any k.The retrieval test significantly favors the proposed weights over category-center weights.Source: B. Validation · p. 9
C. Measuring Interpretability
Interpretability scores depend on the embedding and on how categories and their words are covered in evaluation. I scores above GloVe, while the effects of changing category-word coverage differ across embedding types.
C. Measuring Interpretability
- I is significantly more interpretable than GloVe, whose score is close to the random-embedding baseline.λ controls strictness; the authors propose λ = 5 as an adequate choice corresponding to top-5 category-word retrieval error.Source: C. Measuring Interpretability · p. 9, Figure 8 · p. 10
- Interpretability scores rise as the dataset includes more categories, with I and I* more sensitive to category count than GloVe or random embeddings.The scores are reported across 30, 50, 70, 90, and 110 categories and 40%, 60%, 80%, and 100% category coverage.Source: C. Measuring Interpretability · p. 9
- The effects of within-category word coverage are more complex than those of category coverage.More words may better sample category distributions, but over-abundant categories may weaken semantic correlations and discriminability.Source: C. Measuring Interpretability · pp. 9–10
- As category-word coverage increases, random-embedding scores decrease, GloVe scores increase, and I and I* scores rise through 80% coverage before falling.The authors suggest coverage around 80% may suit interpretability measurement, while noting that non-ideal word subsampling could affect the scores.Source: C. Measuring Interpretability · pp. 9–10
V. DISCUSSION AND CONCLUSION
The paper uses SEMCAT and distribution-based statistics to uncover semantic structure in dense word embeddings and assess their interpretability. It proposes normalized scores for comparing embedding spaces and identifies dataset coverage and labeling as areas for improvement.
V. DISCUSSION AND CONCLUSION
- The framework uses SEMCAT and a statistical method to decompose embedding dimensions and assess interpretability, with qualitative and quantitative checks.SEMCAT contains more than 6,500 words grouped into 110 semantic categories.Source: §V · p. 10
- The proposed interpretability measure is an alternative to the word intrusion test, which relies heavily on human effort while retaining human judgments as its basis.Source: §V · p. 10
- The category weights use Bhattacharya distance, which assumes normally distributed words within each dimension; nonparametric metrics such as Spearman’s correlation are suggested when that assumption fails.The authors report that the GloVe space considered closely follows the normality assumption.Source: §V · p. 10
- Normalized interpretability scores bounded at 100% address the unbounded, nonlinear scale of category weights and facilitate comparisons across embedding spaces.The framework’s comparison depends on the selected category-word dataset.Source: §V · p. 10
- SEMCAT has 110 categories averaging 91 words each; broader coverage and expert labeling by multiple observers could further improve reliability.The authors characterize the categories as comprehensive and report diminishing returns as category counts increase.Source: §V · pp. 10–11
- Evaluating more interpretable embeddings on higher-level tasks and studying their relationship to NLP performance are proposed as future work.Suggested tasks include sentiment analysis, named entity recognition, and question answering.Source: §V · p. 11