PaperlayerView exact revision on arXiv

Source-linked AI summary

Coupled Usage-Sense Processes: Temporal and Attributable Lexical Semantic Change

Haruka Ezoe, Ryohei Hisano

arXiv:2609.30974v1cs.CLstat.ML

TL;DR

Scalar period distances measure how much a word changed but do not show when, by which mechanisms, or through which usages. CUSP builds one marginal-preserving temporal process with compatible change and attribution readouts. Its experiments support recovery, multi-period coherence, and passage-grounded attribution.

  • Problem

    Scalar distances between independently sampled period distributions omit change timing, mechanisms, component transitions, and the usages supporting attribution.

    Source: §1 · p. 1

  • Method

    CUSP couples contextual distributions through latent usage components in one marginal-preserving process, deriving displacement, mechanism and component-pair attribution, word-local modes, and representative passages.

    Source: CUSP: ONE COUPLED TEMPORAL PROCESS,… · p. 2

  • Results

    Experiments support predicted recovery, DWUG scalar validity and decomposition, Janus temporal-profile recovery and coherence, and passage-grounded attribution in U.S. court opinions.

    Source: §4.3 · p. 9, §4.4 · p. 10, §5 · p. 11

  • Takeaways & Limitations

    CUSP makes magnitude, timing, mechanism, movement, modes, and textual evidence compatible views of one lexical history.

    Source: §5 · p. 11

    The recovery analysis assumes known, correctly specified Gaussian-mixture component counts and estimation and optimization conditions.

    Source: §3.4 · p. 7

Abstract

from arXiv · show

Lexical semantic change is usually summarized by a scalar distance between independently sampled period distributions. This measures how much a word changed, but does not reveal when it changed, which mechanisms and component movements carried the change, or which usages support the attribution. We introduce Coupled Usage--Sense Processes (CUSP), which derives these answers from a single marginal preserving temporal process. A hierarchical coupling relates contextual distributions through latent usage components, while Markov composition makes adjacent and longer span correspondences compatible. Displacement operators quantify change magnitude and timing, split variation exactly between movement of component centers and reorganization within components, and attribute it to transported component pairs. Word-local modes resolve distinct directions of change and their activity over time, while representative passages from attributed components ground the analysis in text. Under a Gaussian mixture specialization, we prove parametric recovery of the operators and squared distances. Synthetic experiments support the predicted rate. CUSP remains competitive on English and German DWUG and recovers controlled Janus profiles while maintaining compositionally coherent transport. A large corpus of US court opinions demonstrates transition, mode, and passage attribution in unlabeled natural text. CUSP thus makes magnitude, timing, mechanism, movement, modes, and textual evidence compatible views of one lexical history.

1 Introduction

Standard lexical semantic change evaluations reduce a word’s history to a scalar distance, leaving its timing, mechanisms, transitions, and supporting usages unshown. Prior work addresses individual parts of this problem.

1 Introduction

  • A scalar distance between periods measures how much a word changed but does not show when change occurred or which usages support the attribution.It also does not distinguish movement between usage components from reorganization within them.Source: §1 · p. 1
  • Prior approaches track distributional representations, compare or cluster contextual usages, and measure change using distances, divergences, or optimal transport.Other work tracks sense prevalence and continuity, characterizes temporal patterns, identifies gains and losses in usage mass, or uses embedding axes for interpretation.Source: §1 · p. 1

CUSP: ONE COUPLED TEMPORAL PROCESS, FOUR COMPATIBLE LSC READOUTS

CUSP addresses a gap in interpretable lexical semantic change by building one marginal-preserving temporal process from independently sampled period distributions. Its compatible readouts connect change magnitude and timing to mechanisms, component movements, word-local modes, and representative passages.

CUSP: ONE COUPLED TEMPORAL PROCESS, FOUR COMPATIBLE LSC READOUTS

  • CUSP targets the gap between methods that explain individual parts of lexical change and a single process that supports compatible temporal and attribution readouts.The authors describe interpretability as an unresolved challenge and note that existing methods track senses and attribute change.Source: CUSP: ONE COUPLED TEMPORAL PROCESS,… · p. 2
  • CUSP turns independent period distributions into one marginal-preserving temporal process whose adjacent couplings induce longer-span correspondences.The process does not assert trajectories for individual utterances.Source: CUSP: ONE COUPLED TEMPORAL PROCESS,… · p. 2
  • Hierarchical coupling and Markov composition connect contextual distributions through latent usage components while preserving period marginals and temporal compatibility.Displacement operators provide magnitude and timing, exact mechanism and component-pair attribution, word-local modes, and representative passages.Source: CUSP: ONE COUPLED TEMPORAL PROCESS,… · p. 2
  • Synthetic, DWUG, Janus, and Court experiments support recovery, scalar validity, multi-period coherence, and passage-grounded attribution.Together, these settings retain the conventional measure of how much change occurred while adding compatible answers about when, through which movement, and with what textual evidence.Source: CUSP: ONE COUPLED TEMPORAL PROCESS,… · p. 2

2 Coupled Usage–Sense Processes

CUSP represents each period’s usage distribution as a mixture of latent components, then couples periods into a process with observed marginals. This construction separates movement of component centers from variation within components while keeping longer-span correspondence compositionally consistent.

2.1 Usage distributions in each period

  • Each period’s contextual usages are represented by a mixture of latent usage components, and the coupled process reproduces the observed period distributions.Components are period-local; equal component indices across periods do not themselves establish correspondence.Source: §2 · p. 3, §2.1 · p. 3

2.2 Hierarchical coupling and Markov composition

  • Hierarchical optimal transport couples contextual distributions within candidate component pairs, then allocates prevalence mass subject to the observed period marginals.The adjacent plans share intermediate marginals and are composed to induce non-adjacent correspondences rather than optimizing each period pair independently.Source: §2.2 · pp. 3–4
  • Markov composition yields one process whose period marginals are preserved and whose longer-span correspondences follow the intervening adjacent plans.The construction does not assert trajectories for individual utterances.Source: §2.2 · p. 4

2.3 Full, mean, and deviation processes

  • The coupled contextual process separates into component centers and centered within-component deviations, while the full process retains both.Components can move through embedding space, or their usages can broaden, contract, or rotate around their centers.Source: §2.3 · p. 4

3 Quantifying and attributing semantic change

CUSP’s displacement operators provide compatible measures of change magnitude and timing, with exact accounting by mechanism and transported component pair. Word-local modes distinguish developments within a word, and under a Gaussian specialization the derived operators and squared distances are statistically recoverable.

3.1 Displacement operators and temporal profiles

  • Second-moment displacement operators retain directional information, while their traces give scalar change scores and temporal profiles across adjacent periods.Non-adjacent evaluations inherit the same composed temporal correspondence.Source: §3 · p. 4, §3.1 · p. 5

3.2 Exact mechanism and component transition attribution

  • The squared change decomposes exactly into movement of component centers and within-component variation, then attributes each contribution to transported component pairs.This accounting decomposes the score itself rather than fitting a separate explanatory model; passages from selected source and target components make the attribution inspectable.Source: §3.2 · pp. 5–6

3.3 Word-local temporal modes

  • Word-local modes identify directions organizing a word’s complete history, while adjacent distances locate each direction in time.Unlike vocabulary-wide axes, these modes describe distinct developments within a word.Source: §3.3 · p. 7

3.4 Gaussian implementation and statistical recovery

  • Under Gaussian usage components, quadratic cost, and unrestricted component couplings, closed-form transition terms and distances can be computed for every period pair.The recovery guarantee assumes known, correctly specified component counts and other estimation and optimization conditions; the operators and squared distances stabilize jointly as period samples increase.Source: §3.4 · p. 7
  • Figure 2 tracks finite-sample recovery of component parameters, operators, and squared distances, with the n^-1/2 rate shown for comparison.The figure reports means and 95% confidence intervals over 50 repetitions.Source: Figure 2 · p. 8

4 Experiments

Across synthetic, benchmark, controlled multi-period, and court-opinion settings, the experiments assess CUSP’s recovery, scalar validity, temporal coherence, and passage-grounded attribution.

4 Experiments

  • The lexical experiments use target-token embeddings with Gaussian CUSP, applying a pooled centering and leading-direction removal fixed across words and periods within each dataset.The experiments cover English and German DWUG, Janus, and a balanced Court analysis panel.Source: §4 · p. 8, §4.2 · p. 8, §4.4 · p. 9

4.1 Synthetic mixture processes: finite-sample recovery

  • Synthetic parameter errors decrease with sample size, with operator and squared-distance error slopes whose confidence intervals contain the predicted −1/2 rate.The full operator and squared full-trace errors have log–log slope −0.532; the two squared mode-distance errors have slopes −0.540 and −0.561.Source: §4.1 · p. 8

4.2 DWUG: scalar validity and decomposition

  • CUSP ranks first in English and within 0.012 of the strongest German result; center movement supplies median variation shares of 0.868 and 0.906.The deviation trace remains correlated with human judgments at 0.681 and 0.674, preserving a within-component reorganization signal.Source: §4.2 · pp. 8–9, Table 1 · p. 9

4.3 Janus: controlled multi-period recovery

  • On Janus, CUSP exactly recovers stable profiles, achieves 0.044 TV error for gradual profiles, and identifies all 16 abrupt boundaries; adjacent changes correlate at ρ = 0.954.Its path length separates changed from stable schedules with AUROC 1.000, while composed plans have numerical zero disagreement and preserve endpoint marginals.Source: §4.3 · p. 9, Figure 3 · p. 9

4.4 Court opinions: passage-grounded attribution

  • In court opinions, privacy’s largest standardized transition is 1960→1970, while expectation’s leading modes peak together at that transition.For privacy, center displacement supplies 93.3% of variation and a pair carrying 2.46% of transported mass accounts for 37.1% of contribution; expectation shifts from ordinary anticipation toward the post-Katz standard.Source: §4.4 · p. 10, Figure 4 · p. 10

5 Conclusion

CUSP presents magnitude, timing, mechanism, transported-component attribution, word-local modes, and representative passages as compatible views of one lexical history. The appendices document its notation and proofs, compare its structure with related approaches, and state the identifiability condition for its mixture representation.

5 Conclusion

  • CUSP combines magnitude, timing, exact mechanism and component-pair attribution, word-local modes, and representative passages in one marginal-preserving process.Its Gaussian specialization makes the operators and squared distances tractable and recoverable.Source: §5 · p. 11

A Appendix guide and notation

A.1 Structural comparison with related approaches

  • Table 3 compares approaches across a joint set of capabilities, including compositional coupling, exact attribution, and word-local temporal modes.Its markers distinguish directly provided, related-but-non-equivalent, and absent capabilities.Source: §A.1 · p. 15, Table 3 · p. 17

B Identifiability of the sense mixture representation

  • The mixture representation is identifiable only under a condition making it unique up to permutation of sense labels; label-dependent quantities inherit that permutation ambiguity.The Gaussian-mixture class used for statistical recovery satisfies the stated identifiability condition.Source: B Identifiability of the sense mixture… · p. 15

C Proofs of basic properties of the coupled usage-sense processes

The proofs establish that the coupled process preserves prescribed sense prevalences and conditional contextual distributions at each period. They also establish reconstruction and conditional-centering properties of the process.

C Proofs of basic properties of the coupled usage-sense processes

  • Induction using the transition plans’ second marginal constraint establishes preservation of prescribed sense prevalences at every period.The resulting joint law has the prescribed periodwise usage distributions as its marginals.Source: C Proofs of basic properties of the… · pp. 17–18
  • Induction using the component couplings’ marginal constraints establishes preservation of conditional contextual distributions along positive-probability sense paths.This preserves the component distributions and, consequently, the observed usage distributions period by period.Source: C Proofs of basic properties of the… · pp. 17–18
  • The reconstruction identity follows from the definitions, while conditional centering follows from preservation of the conditional marginals.The proof states that both identities hold.Source: C Proofs of basic properties of the… · pp. 18–19

D Proofs of exact mechanistic and sense-transition attribution

The proofs derive exact mean–deviation and sense-transition decompositions by conditioning on latent sense paths. They also establish finite pseudometric properties for trace and mode-wise semantic distances.

D Proofs of exact mechanistic and sense-transition attribution

  • Conditional centering of the deviation process makes the mean–deviation cross terms vanish in the exact decomposition.The resulting identities apply to the trace and mode-wise readouts.Source: D Proofs of exact mechanistic and… · pp. 19–20
  • Conditioning on latent sense paths and grouping by component pairs yields the exact sense-transition attribution of second-moment operators.The proof applies to both mean and deviation processes.Source: D Proofs of exact mechanistic and… · p. 19

E.1 Validity and properties of the induced semantic distances

  • Trace and mode-wise semantic distances are finite pseudometrics, with zero distance requiring equality only in the corresponding random-vector representation.For a mode-wise distance, the zero condition concerns only projection onto that mode’s direction.Source: §E.1 · p. 21

E.2 Proofs of the decomposition and characterization of the semantic geometry

The proofs establish an orthogonal mode-wise decomposition of total semantic change and a spectral characterization of word-local modes induced by each word’s aggregated mean operator.

E.2 Proofs of the decomposition and characterization of the semantic geometry

  • An orthonormal basis yields the decomposition of total semantic change into orthogonal mode-wise contributions.The proof section says its arguments follow those in [16].Source: §E.2 · p. 22
  • The aggregated mean operator induces a spectral characterization of each word’s local modes.The proof uses the symmetry of the mean operator and the Courant–Fischer variational characterization.Source: §E.2 · p. 22

F Explicit expressions under the Gaussian specialization

Under Gaussian quadratic cost, CUSP gives explicit contextual-coupling and sense-pair second-moment expressions. Dynamic programming computes non-adjacent transition quantities without enumerating latent paths.

F Explicit expressions under the Gaussian specialization

  • Gaussian contextual couplings reduce to quadratic 2-Wasserstein problems with unique affine optimal maps when covariance matrices are positive definite.The minimum coupling cost and the mean and deviation sense-pair second moments have explicit expressions.Source: F Explicit expressions under the… · p. 23

G Dynamic programming for transition quantities

  • Dynamic programming aggregates intermediate components instead of enumerating latent paths, avoiding exponential growth with the number of periods.It computes non-adjacent sense-pair probabilities and cross-period covariance quantities, and hence the deviation components.Source: G Dynamic programming for transition… · pp. 24–25
  • For joint component probabilities, dynamic programming requires O(T^2K^3) time, compared with O(T^3K^T) for naive path enumeration.The corresponding memory costs are O(K^2) for dynamic programming and O(T) for pathwise enumeration.Source: G Dynamic programming for transition… · p. 25
  • Computing cross-period covariance quantities by dynamic programming requires O(T^2K^3d^3) time and O(K^2d^2) memory.The additional computation for deviation components is lower order, O(T^2K^2d^2), and is described as negligible by comparison.Source: G Dynamic programming for transition… · p. 25

H Statistical recovery

Under the Gaussian specialization, the appendix propagates mixture-estimation error through CUSP to operator and distance recovery. Synthetic experiments show the predicted rate and strong allocation of path energy to planted mechanisms.

H Statistical recovery

  • The recovery analysis assumes the true number of Gaussian mixture components is known, with a separate model-selection result supporting consistent component-number recovery.The subsequent error analysis conditions on the event of correct component-number recovery.Source: H Statistical recovery · pp. 27–28
  • Under the Gaussian specialization, estimation errors propagate through the coupling, sense transport plans, and second-moment operators to yield recovery rates for induced distances.The analysis conditions on correct component-number recovery, which holds with probability tending to one as sample size increases.Source: H Statistical recovery · pp. 27, 28, 33

I.1 Synthetic mixture process and supplementary recovery

  • With n = 1600 usages per period and 20 repetitions, CUSP assigns 99.89%, 98.95%, and 99.95% of path energy to planted center, covariance, and prevalence mechanisms, respectively.The remaining 1.05% mean allocation in the covariance-only process is finite-sample estimation error.Source: §I.1 · p. 34
  • Operator, trace-distance, and mode-distance error intervals all contain the n^-1/2 reference rate in the synthetic experiment.The experiment uses a three-period, two-dimensional process and 50 repetitions across sample sizes from 100 to 1,600 usages per period.Source: §I.1 · p. 34

I.2 DWUG implementation and decomposition

The supplementary lexical evaluations examine DWUG decomposition, controlled Janus profiles, and Court-corpus patterns. Their results support distinct claims while retaining boundaries on model recovery and interpretation.

I.2 DWUG implementation and decomposition

  • For lexical experiments, regularized GMMs are approximations and the Gaussian recovery theorem does not guarantee recovery on normalized lexical data.The decomposition remains exact for each fitted process.Source: §I.2 · p. 36
  • The DWUG mean trace remains close to the full trace, while the deviation trace is weaker but associated with human scores and exposes within-component reorganization.The decomposition identifies change that the scalar full trace would otherwise conceal.Source: §I.2 · p. 36

I.3 Janus construction and supplementary results

  • CUSP recovers planted Janus temporal profiles and provides compositionally coherent transport, with pairwise-plan inconsistency concentrated in gradual replacement.The experiment does not establish uniform superiority over pairwise optimal transport.Source: §I.3 · p. 37

I.4 Court corpus construction and fitting

  • Court components describe recurring usage structure rather than a unique inventory of linguistic senses, and the component cap determines fitted resolution.BIC selects the maximum K=5 in 471 of 480 fits; alternative caps may change assignments and the mean–deviation allocation.Source: §I.4 · p. 38
  • In the Court analysis, privacy ranks among the top five lemmas by enrichment and top two by secondary-mode temporal separation across preprocessing settings.Its largest peak falls in either 1950→1960 or 1960→1970, so the peak decade varies across settings.Source: §I.4 · p. 38

I.5 Court rankings and passage analysis

The Court analysis numerically selects component movements before retrieving representative source and target passages. Its examples show how component-level attributions distinguish legal usages and how word-local modes capture different temporal profiles.

Passage analysis

  • Representative passages are retrieved only after numerical selection, assigned to fitted components by maximum posterior probability, and ranked by center proximity and lemma matching.CourtListener identifiers link excerpts to their source opinions; the selected source and target passages are representative endpoints, not observed sentence pairs.Source: §I.5 · p. 39

Peak transition attribution by maximum enrichment

  • Emax identifies a peak-transition component movement whose contribution share is most disproportionate to its transported-mass share.The score is a per-unit-mass attribution statistic, not a significance measure; rare component pairs can have uncertain estimated displacements.Source: §I.5 · p. 39, Table 9 · p. 39
  • Privacy’s leading selected transition moves from Prosser-style tort classification toward post-Katz expectation-of-privacy language.The preceding decade’s leading pair moves from privacy as a direct personal wrong toward the Prosser component.Source: §I.5 · pp. 39–40
  • The selected search transition moves from a legally required effort to obtain work toward physical inspection.The work-search source is retained because it is part of a legal disability test, not incidental narration.Source: §I.5 · p. 40
  • Standing separates an adjectival property sense from a jurisdictional doctrine, while warrant and trespass show shifts between distinct legal constructions.Warrant moves from licensing a conclusion to a judicial instrument; trespass redistributes between distinct legal constructions without assigning the shift to a landmark decision.Source: §I.5 · p. 40

Temporal separation of word-local modes

  • Secondary-mode temporal separation prioritizes passage analysis when the two leading modes peak at different transitions; synchronized profiles receive no S2 rank.The score prioritizes analysis and does not estimate the model, test significance, or determine whether change occurred.Source: §I.5 · p. 40, Table 10 · p. 41

Privacy modes

  • Privacy’s modes recover staggered tort, constitutional, informational, and search usages rather than one undifferentiated event.Mode 2 peaks at 1950→1960; modes 3 and 4 peak at 1960→1970, while mode 1 remains strong into 1970→1980.Source: §I.5 · pp. 40–41

Expectation modes

  • The three leading expectation modes carry 48%, 26%, and 18% of mean-path energy, and all peak at 1960→1970.Their selected passages move from ordinary anticipation toward the post-Katz Fourth Amendment standard.Source: §I.5 · p. 41

Related papers

Loading 2609.30974v1…