PaperlayerView exact revision on arXiv

Source-linked AI summary

Compound interpretation is based on analogy

Tian Shen, Harald Baayen

arXiv:2610.01688v1cs.CL

TL;DR

Compound meaning must be related to constituent meaning, but whether this is best captured by a global transformation or local analogy remains an open question. The paper proposes CAM, which uses constituent embeddings and family shift vectors, and compares it with CAOSS on Mandarin compounds. CAM generally predicts compound embeddings more accurately, with an exception for held-out three-character compounds constrained by small and imbalanced constituent families.

  • Problem

    Compound meanings vary in predictability from constituent meanings, and the paper asks whether semantic-vector accounts are better characterized by a global transformation or local analogical generalization.

    Source: §1 · pp. 3–4

  • Method

    CAM adds each constituent embedding to the centroid of its compound-family shift vectors and averages the resulting transcended meanings to predict the compound embedding.

    Source: §6 · p. 31

  • Results

    CAM outperformed CAOSS on training data and held-out two- and four-character compounds, while CAOSS was more accurate on held-out three-character compounds.

    Source: §6 · p. 32

  • Takeaways & Limitations

    Two-character compound meanings cannot be predicted with sufficient accuracy from constituent meanings alone, limiting obligatory-decomposition accounts’ reliance on a general prediction algorithm.

    Source: §6 · p. 33

    For compounds with constituent hermits, analogical generalization becomes impossible.

    Source: §4.5 · p. 22

Abstract

from arXiv · show

How compound meanings are best predicted from constituent meanings remains a central question in computational models of lexical semantics. Comparing different computational models provides a way to evaluate alternative accounts of how semantic information is combined during compound comprehension. We propose a new model, the Compound Analogy Model (CAM), that predicts a compound's embedding by adding its constituent embeddings together with the average shift vectors of the two constituents' compound families. The resulting model is parameter-free and exploits local analogical structure in the semantic space. We evaluated CAM against the CAOSS model on Mandarin Chinese compounds. CAM consistently achieved higher prediction accuracy than CAOSS on both training and held-out data, with the exception of three-character compounds, for which analogical generalization is constrained by both small constituent families and a pronounced imbalance in family size between the two constituents. The advantage of CAM remained when evaluation was based on frequency-defined train-test splits that better approximate generalization from familiar to novel compounds. To assess the cognitive plausibility of the two models, we further examined whether model-derived semantic measures predict visual lexical decision latencies for two-character compounds. Predictors derived from CAM provided improved prediction for response latencies compared to predictors derived from the CAOSS model. These findings indicate that compound meaning is better characterized as local analogical generalization than as the application of a learned global linear transformation, and demonstrate that analogical semantic structure provides a cognitively plausible basis for compound comprehension.

1 Introduction

Compound interpretation requires relating constituent meanings to whole-word meanings despite wide variation in predictability. This study tests whether semantic-vector accounts should use a global transformation or local analogical generalization.

1 Introduction

  • Compound meanings vary in how predictably they follow from constituents, making compounds an informative test of semantic composition.The challenge remains whether compound meaning reflects a learned global transformation or local analogical generalization.Source: §1 · pp. 3–4
  • Only two of 600 English compounds received unanimous assignment to one of 16 semantic relations, and the median compound received more than five relations.This suggests manually crafted relational categories cannot fully capture semantic composition.Source: §1 · p. 4
  • CAOSS treats compound interpretation as a general function mapping constituent embeddings to a compound embedding, a computational account of semantic composition and reader similarity judgments.Its proposed single semantic operation parallels accounts of derivation.Source: §1 · pp. 4–5
  • Local analogical generalization uses similarity neighborhoods rather than a universal rule, potentially capturing reliable local patterns while limiting noise from atypical formations.The study compares CAOSS and CAM on training and held-out data, then evaluates model-derived measures against visual lexical decision latencies.Source: §1 · p. 5

2 Compounding in Mandarin Chinese

Mandarin Chinese relies heavily on compounding, and its characters can generally be recognized as constituents within compounds. Character polysemy and varied compound structures make constituent-to-compound meaning prediction challenging.

2 Compounding in Mandarin Chinese

  • Compounding accounts for approximately 70% of existing Mandarin words and around 95% of newly coined words.Chinese characters mostly correspond to spoken syllables, and readers distinguish words despite texts lacking spaces.Source: §2 · p. 6
  • Characters can be highly polysemous, as 生 functions as a verb, adjective, or noun across different compounds.The passage notes that this amplifies the constituent-sense challenge and highlights the potential importance of local analogical structure.Source: §2 · p. 6
  • Mandarin compounds include modifier-head, coordinate, verb-object, subject-predicate, and verb-resultative structures.The position of functional heads is described as notably free compared with English modifier-head compounds.Source: §2 · p. 6
  • Earlier Mandarin studies examined CAOSS-derived transparency measures and a transformer-based NOTCH model, but NOTCH addresses a different compositionality question.The paper instead focuses on simpler models using non-contextualized embeddings from the same semantic space.Source: §2 · pp. 6–7

3 The models

The section introduces CAOSS, which predicts compound meanings through a learned linear combination of constituent embeddings, and CAM, which uses local analogical structure in constituent families. CAM estimates compound meanings with constituent embeddings adjusted by average family shift vectors.

3.1 The CAOSS model

  • CAOSS predicts compound embeddings as a weighted sum of constituent embeddings, with weights estimated using least-squares multivariate multiple regression.The model concatenates the constituent embeddings and maps them to compound embeddings with a learned transformation matrix.Source: §3.1 · p. 7

3.2 The Compound Analogy Model (CAM)

  • CAM predicts a compound meaning from its constituent embeddings and the average shift vectors of their compound families.The model averages the two constituent-based adjusted embeddings to constrain the predicted vector's length.Source: §3.2 · pp. 10–11
  • A constituent-family shift-vector centroid estimates that constituent’s prototypical semantic contribution in its position within compounds.CAM combines these position-specific adjustments with constituent embeddings to form the predicted compound representation.Source: §3.2 · p. 11
  • CAM uses local semantic shifts that linear transformations cannot implement, so CAOSS is not a general case of CAM.Source: §3.2 · p. 14

4 Computational experiments

Across the full Mandarin dataset, CAM outperformed CAOSS on training and held-out accuracy, and its advantage persisted when low-frequency compounds were held out. The length-based analyses qualified that pattern: CAOSS led on held-out three-character compounds, while sparse or imbalanced constituent families constrained analogy.

4.1 Materials

  • The evaluation used Chinese Lexical Database entries and Tencent embeddings, excluding non-segmentable three-character words and 31 four-character sequences that could not be reliably segmented.Three-character boundaries were assigned using a two-character-word lookup and frequency when both adjacent sequences existed; the analyses included 33,844 two-, 6,008 three-, and 2,274 four-character words.Source: §4.1 · pp. 14–15

4.2 Method

  • The models were compared using nearest-neighbour retrieval accuracy across 30 random 90/10 train-test splits, with compound-vector L2-norms examined separately.Accuracy counted a prediction as correct when its gold target had the highest Pearson correlation among gold-standard embeddings.Source: §4.2 · p. 16

4.3.1 Full data set

  • CAM's mean training advantage was 0.26, and held-out accuracy was 0.17 vs. CAOSS's 0.16; on low-frequency held-out compounds, accuracy was 0.12 vs. 0.09.The frequency split also favored CAM on familiar training compounds (0.45 vs. 0.19).Source: §4.3.1 · pp. 16–17, Figure 6 · p. 17, Figure 7 · p. 18

4.3.2 Performance by word length

  • For held-out three-character compounds, CAOSS accuracy was 0.53 vs. CAM's 0.47, while CAM outperformed CAOSS on held-out two- and four-character compounds.Training accuracy was higher for CAM at all three lengths; overall test accuracy rose with word length, while held-out variability also increased.Source: §4.3.2 · p. 18, Figure 8 · p. 19
  • Under the frequency-based split, only two-character compounds showed a significant CAM advantage on low-frequency held-out data; noun-only analyses showed a similar three-character exception.For two-character nouns, CAM test accuracy was 0.20 vs. CAOSS's 0.17; four-character noun test accuracy was 0.75 vs. 0.57, with substantial variance.Source: Figure 9 · p. 19, §4.3.2 · pp. 19–20, Table 2 · p. 20

4.4 Vector length

  • Predicted compound-vector L2-norms correlated more closely with observed norms for CAM than CAOSS (0.44 vs. 0.33), and CAM's absolute norm divergences were smaller.CAOSS predicted compound vectors were substantially shorter than their gold-standard counterparts.Source: §4.4 · p. 21, Figure 11 · p. 21

4.5 Discussion

  • CAOSS's global transformation was diagonally dominant for two-character compounds, while off-diagonal weights contributed more for longer compounds; CAM better captured training-data structure.The diagonal elements contributed roughly 72.8% of W's Frobenius norm for two-character compounds, declining to 65.4% and 59.1% for three- and four-character compounds.Source: §4.5 · pp. 22–24, Figure 12 · p. 23
  • Three-character compounds had the greatest constituent-family asymmetry (index 1.87), and their very small families constrain analogical generalization.For held-out three- and four-character compounds with hermit constituents, CAM cannot calculate shift vectors and reduces to summing constituent vectors.Source: §4.5 · pp. 22–23

5 Predicting visual lexical decision response latencies

CAM-derived predictors provided an advantage over CAOSS predictors for visual lexical decision latencies, although the authors characterize CAM as a competitive alternative given sensitivity to modeling choices. The analyses also show that CAOSS proximity measures may reflect predicted compound length, whereas CAM proximities interact in predicting latencies.

5.1 Data

  • 8,752 two-character compounds from 504 native Chinese speakers entered the latency analysis, which used generalized additive models and three model-derived predictors per model.The sample was restricted to items shared with the computational modeling dataset after excluding words with error rates above 30%.Source: §5.1 · pp. 24–25

5.2.1 CAOSS model

  • For CAOSS, cosine similarity was associated with shorter latencies at high right-constituent family sizes, while the word-frequency effect levelled off at higher frequencies.At low right-constituent family sizes, cosine changes were associated with small, non-monotonic latency changes.Source: §5.2.1 · pp. 25–26, Figure 13 · p. 27
  • CAOSS proximity measures were interpreted as reflecting predicted compound-vector length rather than semantic similarity alone.Replacing the proximity measures with predicted length reduced AIC from -11570.10 to -11573.05; using gold-standard length reduced it to -11631.9.Source: §5.2.1 · pp. 26–28

5.2.2 CAM

  • For CAM, the cosine-by-family-size effect crossed over, and latencies decreased most when both constituent proximity measures were large.The model included tensor-product smooths for these interactions.Source: §5.2.2 · p. 28, Figure 15 · p. 29

5.2.3 CAOSS and CAM compared

  • CAM assigned greater variable importance to first-constituent proximity than to second-constituent proximity, reversing CAOSS’s pattern; second-constituent family size mattered more for CAOSS.Word frequency had the greatest variable importance in both models.Source: §5.2.2 · p. 28, §5.2.3 · p. 30
  • AIC -11597.45 for CAM versus -11570.10 for CAOSS; the authors describe CAM as a competitive alternative for predicting visual lexical decision latencies.They caution that results can vary with word frequencies, embeddings, and covariates.Source: §5.2.1 · p. 25, §5.2.3 · p. 30
  • Adding gold-standard compound-vector length improved both models’ fit, while all other terms remained significant, indicating value beyond that length measure.AIC changed from -11,570.10 to -11,672.38 for CAOSS and from -11,597 to -11,695.46 for CAM.Source: §5.2.3 · p. 30

6 General Discussion

CAM generally outperformed CAOSS on Mandarin compounds and provided better prediction of visual lexical decision latencies, but its analogical generalization weakens when constituent families are small. The findings support local analogy as a basis for compound interpretation while leaving limits on prediction from constituents alone, especially for novel compounds.

6 General Discussion

  • CAM and CAOSS embody different assumptions: local regularities among lexical neighbors versus a learned transformation across compounds.CAM adds each constituent's family shift-vector centroid to its embedding, then averages the resulting constituent meanings.Source: §6 · p. 31
  • CAM outperformed CAOSS on training data and on held-out two- and four-character compounds, while CAOSS did better on held-out three-character compounds.CAM is non-parametric, whereas CAOSS estimates an 80,000-parameter linear transformation.Source: §6 · p. 32
  • Small constituent families can reduce CAM's analogical basis to a single exemplar, while CAOSS uses more off-diagonal weights for longer, more transparent compounds.For held-out longer compounds, CAM's prediction reduces to the mean of the two constituent embeddings.Source: §6 · p. 32
  • Proximity measures may largely reflect predicted-vector length rather than semantic transparency; the only reported transparency evidence was a small cosine-similarity effect.Replacing the proximity measures with predicted-embedding L2 norm improved model fit.Source: §6 · pp. 32–33
  • A GAM with CAM-based predictors outperformed one with CAOSS-based predictors by more than 20 AIC units when predicting visual lexical decision latencies.The CAM-based model included constituent-proximity measures and the cosine similarity of predicted and observed compound embeddings.Source: §6 · p. 33
  • Two-character compounds were predicted with 25% accuracy by CAOSS and 50% by CAM, so obligatory decomposition cannot rely on a general constituent-to-compound prediction algorithm.A character n-gram network predicted embeddings from written forms with 100% accuracy, and adding its compound-embedding L2 norms decreased GAM AIC by about 96 units.Source: §6 · p. 33
  • Analogical shift-vector clusters may capture semantic relations such as “made of,” but the basis for analogy can shrink to single exemplars in compounds with more than two constituents.The account also cautions that related analogical models face this small-exemplar constraint.Source: §6 · p. 34
  • Local analogy may provide a gist for novel compounds, but their interpretation is expected to depend heavily on context; the observed two-character structure was too local for robust generalization to unseen compounds.The authors also estimate that their 33,800-word dataset may exceed many educated speakers' lexical knowledge.Source: §6 · pp. 34–35
Loading 2610.01688v1…