PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamics of meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala

Nevidu Jayatilleke, Nisansa de Silva

arXiv:2609.08609v1cs.CL

TL;DR

Diachronic semantic change in low-resource Sinhala is difficult to evaluate because historical data are sparse and embedding spaces require cross-period comparison. The paper builds an enhanced corpus and combines static embedding alignment with contextualised Llama-FT analysis and leave-one-out diagnostics. It finds that OP offers more stable neighbourhood tracking and that observed contextual semantic drift is concentrated in a smaller subset of high-impact usages.

  • Problem

    Diachronic semantic-change evaluation remains comparatively underexamined in data-scarce settings, where historical corpora are limited.

  • Method

    The study enriches a Sinhala diachronic corpus, aligns century-specific Word2Vec and FastText spaces with SMA and OP, and applies LOO-based semantic impact pruning to Llama-FT embeddings.

  • Results

    The average Llama-FT global centroid distance is 0.0295, falling to 0.0272 after high-impact-instance removal, an 8% drift reduction; OP also provides superior stability for temporal similarity dips.

  • Takeaways & Limitations

    Semantic drift in this analysis is driven by selective contextual innovations rather than uniform shifts across all occurrences, supporting instance-focused diachronic analysis.

  • Takeaways & Limitations

    Historical data scarcity weakens century-wise Word2Vec models and may make Procrustes alignment less reliable than with fine-tuned FastText models.

Abstract

from arXiv · show

Tracking semantic change in low-resource languages across extensive historical timelines presents significant challenges due to data scarcity and the limitations of static embedding alignments. This study investigates the diachronic evolution of the Sinhala language from the 13th to the 20th century using a multi-stage computational framework. We first align century-specific Word2Vec and FastText embeddings using Similarity Matrix Based Alignment (SMA) and Orthogonal Procrustes (OP) techniques, finding that OP alignment provides more stable neighbourhood tracking for identifying temporal similarity dips. To move beyond aggregate measures, we introduce a Bidirectional Semantic Impact Pruning approach using contextualised embeddings from a fine-tuned Llama-3.1-8B. By applying Leave-One-Out (LOO) diagnostics, we attempt to isolate influential sentences to distinguish between systemic semantic shifts and transient polysemic expansion. Our results show that semantic drift in the fine-tuned Llama-3.1-8B is not evenly distributed across all usages. Instead, a significant part of the change is driven by a smaller set of high-impact contextual instances, rather than gradual and uniform change across all occurrences. This work provides a preliminary framework for diachronic analysis in low-resource contexts, highlighting the trade-offs between model sensitivity and data availability.

1 Introduction

Language changes over time as meanings vary across discourse and historical contexts. This study examines such diachronic semantic change in low-resource Sinhala using an enhanced corpus and century-based embedding analysis.

  • Meaning prevalence varies with genre, register, and historical context, producing diachronic semantic changes such as “gay” shifting from “cheerful” to “homosexual.”
  • Sinhala is an Indo-European language with a literary history spanning several millennia and about 16 million first-language speakers.
  • Sinhala is classified as a lower-resourced language and has undergone substantial historical evolution.
  • The study uses SiDiaC-v.2.0, creates a lemmatised and POS-tagged SDC version, and evaluates static and contextualised diachronic embeddings across centuries.

2 Related Works

Prior diachronic semantic-change research has developed sophisticated models, but data-scarce settings remain comparatively underexamined. Existing work addresses temporal smoothing, contextualised sense discovery, and small-corpus evaluation through different modelling strategies.

  • Research has advanced from statistical laws of semantic change to unsupervised detection frameworks, while data-scarce settings remain a significant gap.
  • Hamilton et al. distinguish conformity, where frequent words change slowly, from innovation, where polysemous words change rapidly.
  • Global embedding shifts capture regular linguistic drift, whereas local-neighbourhood changes better detect irregular cultural shifts.
  • SCAN and Dynamic Bernoulli Embeddings model semantic evolution continuously to smooth sparse historical representations.
  • Other studies investigate semantic change with contextual clustering, small corpora, and dynamically constrained embeddings under severe data scarcity.

3 Methodology

The methodology constructs and enriches a century-filtered Sinhala corpus, trains static and contextualised embeddings, and maps their spaces for cross-century comparison. It combines relational similarity and Procrustes alignment to address morphological variation, stochastic rotations, and sparse data.

  • 3.1 Creation of SDC: The study uses 59 dated books containing 67,005 words from SiDiaC-v.2.0, spanning the 5th through 20th centuries.
  • 3.1 Creation of SDC: The enhanced SDC adds morphological lemmas and POS tags using SinMorphy, an improved TnT tagger, and Sinling.
  • 3.2 Vectorisation: The 5th, 12th, and 16th centuries were excluded for insufficient counts; removing the 16th century increased shared words across periods from 80 to 210.
  • 3.2 Vectorisation: Word2Vec and FastText were trained or fine-tuned by century, while contextual embeddings came from the best-performing fine-tuned Llama-3.1-8B model.
  • 3.3 Computational Semantic Mapping: The framework aligns century-specific Word2Vec and FastText spaces with Similarity Matrix and Orthogonal Procrustes methods.
  • 3.3 Computational Semantic Mapping: Alignment is needed because stochastic training creates arbitrarily rotated, non-identifiable embedding spaces, even though within-space similarities remain comparable.
  • 3.3 Computational Semantic Mapping: A centroid lemma representation averages constituent surface-form vectors, while 261 common lemmas provide a relational cross-century reference frame.
  • 3.3.2 Orthogonal Procrustes Based Alignment: Neural OP outperforms Analytical OP for FastText, producing lower disparity and distance errors and better manifold preservation.

4 Evaluation of Semantic Change

The evaluation combines dynamic thresholding and neighbourhood comparison for static embeddings with LOO-based contextual diagnostics. OP alignment is more stable than SMA, while contextual drift is concentrated in high-impact usages rather than distributed uniformly.

  • Statistical Neighbourhood Analysis: Semantic-change candidates are flagged when similarity falls below the dynamic threshold µ − (k × σ), with k set to 0.8.The cutoff adapts to each temporal pair instead of applying one static cosine threshold.
  • Statistical Neighbourhood Analysis: 16.19%–20.57% of instances were flagged across four pipelines, averaging 18.36% and remaining within five percentage points of the theoretical 21.19%.The observed proportions support k = 0.8 as a stable, near-canonical vocabulary fraction rather than a pipeline-specific artefact.
  • Statistical Neighbourhood Analysis: OP alignment was more robust than SMA because SMA’s global relational calculation can be distorted by locally changing lemmas.The study evaluated Word2Vec and FastText with SMA and two OP variants across 15 temporal alignments.
  • Statistical Neighbourhood Analysis: Word2Vec showed more frequent temporal similarity dips and lower stability than FastText across the aligned semantic spaces.FastText’s high similarity may partly reflect fine-tuning on pretrained models to compensate for the small corpus.
  • Bidirectional Semantic Impact Scoring: LOO masking identifies primary informer sentences and removes them to separate observed global distance from persistent semantic shift.The resulting drift reduction estimates the share attributable to documented contextual innovations.
  • Bidirectional Semantic Impact Scoring: The average Llama-FT centroid distance fell from 0.0295 to 0.0272 after removing high-impact instances, an 8% drift reduction.Among 112,945 instances, 21,738 were identified as drifted, indicating uneven change across occurrences.
  • Bidirectional Semantic Impact Scoring: The lemma guna shifted from predominantly dynastic and religious virtue-praise toward Buddhist compassion-praise and courtly-poetic praise of feminine beauty.The example illustrates divergent contextual trajectories across the 14th and 15th centuries.
  • Bidirectional Semantic Impact Scoring: Overall, the contextual analysis supports selective contextual innovations rather than uniform semantic shifts across all instances.The study also reports small-scale manual validation across all 30 ordered permutations of the six centuries.

5 Conclusion

The study establishes a multi-method framework for Sinhala diachronic semantics. Static alignment analysis favours OP stability, while contextual LOO pruning isolates influential sentences to distinguish systemic change from polysemic expansion.

  • Framework: The framework aligns century-specific Word2Vec and FastText models with SMA and OP techniques for diachronic semantic analysis.Statistical neighbourhood analysis evaluates the resulting temporal spaces.
  • Static Embeddings: OP alignment showed superior stability for identifying temporal similarity dips in the Sinhala lexicon.This conclusion concerns the static embedding evaluation.
  • Contextual Analysis: Bidirectional Semantic Impact Pruning uses Llama-FT contextual embeddings and LOO diagnostics to identify sentences responsible for observed semantic drift.The approach extends analysis beyond aggregate centroid-based measures.

Limitations

The study’s scope and outcomes are constrained by sparse historical data and limited historical-language tooling. These constraints affect embedding reliability, morphological analysis, and part-of-speech labelling.

  • Data Scarcity: Sparse historical corpora weakened century-wise Word2Vec models and may make Procrustes alignment unreliable relative to fine-tuned FastText.Data scarcity also possibly affected the strength of expanded-vocabulary embeddings.
  • Data Scarcity: Continual pretraining did not improve transformer perplexity or BPB scores, possibly because data scarcity weakened expanded-vocabulary embeddings.The limitation is stated as a possible effect rather than a definitive causal finding.
  • Lemmatisation: SinMorphy may not have been trained on historical Sinhala, limiting confidence in lemmatisation for historical text.The paper links this gap to the absence of robust morphological analysers.
  • POS Tagging: The lack of high-accuracy historical Sinhala POS taggers required a hybrid system that may introduce modern grammatical bias.That bias could affect morphological labelling in medieval and early modern corpus sections.
  • Research Scope: The corpus literature review identifies a scarcity of diachronic semantic-change studies for lower-resource languages.SiDiaC-v.2.0 is described as one of only three corpora from language class 3 or below studied for semantic change.

B.2 SiDiaC-v.2.0

SiDiaC-v.2.0 is the largest comprehensive diachronic Sinhala corpus to date, with a precisely dated subset supporting historical analysis. Its processing includes OCR, genre organisation, native-speaker post-processing, and special tokens for corpus structure and poetic suffixes.

  • Corpus Scope: SiDiaC-v.2.0 contains 244,204 tokens across 186 literary works, including 59 precisely dated documents with 67,005 words spanning the 5th–20th centuries.The dated subset enables accurate historical analysis despite the full corpus’s publication-date range of 1800–1955 CE.
  • Processing: Google Document AI OCR was used for historical Sinhala text extraction, including text modernisation and morpheme segmentation.The corpus retains a two-level genre classification beginning with Fiction and Non-Fiction.
  • Corpus Organisation: The corpus further categorises texts into Religious, History, Poetry, Language, and Medical domains.Table 5 reports resource distribution across genres and written-time periods.
  • Post-processing: Native-speaker post-processing removed Pali, Sanskrit, and English code-mixed content and corrected formatting issues.Special tokens mark sentence boundaries and preserve Sinhala poetry’s distinctive suffix structure.

C.1 Lemmatisation

The study lemmatises the Sinhala corpus with SinMorphy and resolves morphological and POS ambiguities through a priority-based combination of TnT and Sinling. The resulting pipeline includes fallback handling for unknown or invalid analyses.

  • Lemmatisation: The pipeline processes tokenised sentences through a Selenium-based browser instance because SinMorphy is available only through a web interface.Adaptive retries reduce the effects of latency and timeouts.
  • Lemmatisation: SinMorphy performs rule-based morphological analysis and lemmatisation without deep learning or contextual information.It uses a finite-state transducer and a guesser for unknown words.
  • POS Tagging: TnT uses a second-order HMM with Viterbi decoding and linear interpolation smoothing, trained on news and official-document corpora.The implementation replaces exhaustive decoding with beam search for more efficient tagging.
  • POS Tagging: 23,180 words were labelled ‘UNK’ because the lexicon lacked words spanning different historical periods.Sinling was used to identify POS tags for words receiving multiple tags from SinMorphy.
  • POS Tagging: Ambiguous POS assignments are resolved first with TnT and then with Sinling, after mapping outputs to seven standard categories.If neither tagger matches a candidate, the algorithm selects the first analyser output.

D.1 Continual Pre-training (CPTing)

The study applies continual pre-training with vocabulary expansion across causal and masked language models. The adaptation strategies differ by architecture, stabilising causal-model embeddings while fully optimising the masked-language-model representation.

  • Model selection: CPTing evaluates Llama-3.1-8B and SinLlama as causal models alongside XLM-R as a masked language model.Out-of-vocabulary terms are extracted from the full SiDiaC-v.2.0 corpus for dynamic vocabulary expansion.
  • Causal language models: Causal models integrate base or expanded tokenisers by initialising new embedding weights with the average existing embedding vector.The approach targets structural stabilisation within the transformer blocks.
  • Masked language model: XLM-R uses end-to-end full-parameter optimisation to align expanded token structures with its multilingual representations.Its standard embedding layers permit direct structural modification.
  • Masked language model: XLM-R is expanded to 307,386 tokens after adding 57,384 tokens and trained with 15% masking while all encoder layers remain unfrozen.The procedure optimises the bidirectional contextual weights globally.

D.2 Intrinsic Evaluation

Intrinsic evaluation uses sliding-window corpus metrics to compare model processing efficiency across tokenisation settings and architectures. BPB provides a tokeniser-agnostic comparison, while the results show that vocabulary expansion raises evaluation difficulty and that Llama-FT is selected for later bidirectional analysis.

  • Evaluation metrics: BPB is derived from corpus negative log-likelihood and raw UTF-8 byte length, enabling tokeniser-agnostic comparison across architectures.Perplexity remains dependent on vocabulary size because it is normalised per token.
  • Results: Expanded vocabularies produce higher perplexity and BPB than corresponding standard models in the CPT experiments.The authors attribute the perplexity increase to broader next-token choice and greater probability-distribution uncertainty.
  • Results: The approximately 241k-word SiDiaC-v.2.0 corpus is insufficient to prevent increased BPB after vocabulary expansion.The result indicates a need for more data for this task.
  • Results: Llama-3.1-8B has the strongest baseline among standard decoder-only architectures across perplexity and BPB.The instruct variant performs worse than the base model on both metrics.
  • Results: XLM-R achieves the second-best BPB score, although its masked-language-model objective does not produce traditional perplexity.The analysis concludes that Llama-FT is the model selected for bidirectional semantic analysis.

E.1 Orthogonal Procrustes Alignment

The OP alignment analysis compares Word2Vec and FastText using Procrustes disparity and rotational distance across historical reference-to-target mappings. FastText improves in alignment precision with more data, whereas Word2Vec shows worsening metrics despite increased data volume.

  • Evaluation: OP alignment is evaluated with Procrustes disparity and rotational distance for analytical and neural alignment approaches.The scores quantify mappings from a reference century to a target century.
  • FastText: FastText Procrustes Disparity falls from 68.38 at 4,707 tokens in the 18th century to 49.62 at 17,295 tokens in the 20th century.This follows the expected pattern of improved structural stability with greater data density.
  • Word2Vec: Word2Vec Procrustes Disparity rises from 140.86 to 166.86 as data increases from the 18th to the 20th century.Its Rotational Distance also increases from 8.77 to 9.35 across the same comparison.
  • FastText: FastText maintains a Rotational Distance of approximately 19.72 across centuries while its disparity decreases with additional data.The authors relate this stability to pretrained subword geometric constraints.
  • Interpretation: FastText’s higher rotational distance reflects alignment of a rigid pretrained subword space, whereas Word2Vec can rotate locally because it is learned from scratch.The larger global rotation preserves FastText’s internal structure and yields higher alignment precision.

F.1 Statistical Neighbourhood Analysis

The analysis identifies recurring diachronic shifts in Sinhala lemmas, often from spiritual, religious, or elite contexts toward physical, everyday, technical, social, or poetic meanings. Examples span changes in religious imagery, intellectual and material domains, and abstract philosophical usage.

  • ‘පුරැෂ’ shifts from Buddhist supernatural attributes in the 18th century to ordinary mortality and suffering by the 20th century.The later semantic field centers on physical deterioration and death.
  • ‘චන්šයා’ moves from a religious metaphor for spiritual light in the 14th century to courtly and romantic symbolism by the 15th century.Later neighbours associate the moon with beautiful women and royal identity.
  • Several lemmas shift from spiritual or learned meanings toward practical and everyday domains, including ‘සැප’, ‘මහත්’, ‘භාව’, and ‘මාර්ග’.These changes include movement from religious liberation to physical well-being, intellectual achievement to quantity, spiritual ideas to conduct, and moral direction to physical or textual usage.
  • ‘රෑප’, ‘සිත’, ‘උපද’, and ‘අග‡’ likewise develop later associations with philosophical, pathological, material, or other abstract semantic domains.The examples indicate that diachronic change can reorganize a lemma’s neighbourhood across bodily, worldly, technical, and doctrinal contexts.
  • Other examples show movement from physical or transactional meanings toward figurative, social, or supernatural fields, including ‘රන්’, ‘මල්’, and ‘බලෙයන්’.The documented changes connect everyday objects or physical action with poetic imagery, religious devotion, and supernatural or social associations.

F.2 Bidirectional Semantic Impact Scoring

Llama-FT embeddings show largely stable core semantic structures, while observed drift is concentrated in a smaller set of high-impact contextual usages. Divergence is especially pronounced in 18th-century comparisons and in specific lemmas whose surrounding registers changed across periods.

  • Semantic drift remains relatively limited overall, with most changes concentrated in a smaller subset of high-impact contextual usages rather than uniform lexical restructuring.Across 261 consistent lemmas, global and persistent shifts remain close, indicating substantial continuity alongside localized innovations.
  • 0.0340 is the highest average global distance, observed for 18 →19, with a persistent shift of 0.0299.All ten permutations involving the 18th century have global distances of at least 0.0259, above the crosspermutation mean of 0.0238.
  • 0.0154 is the lowest centroid distance, observed for 20 →19, followed by 20 →13 at 0.0158 and 20 →15 at 0.0161.These low distances indicate stronger semantic continuity between the 20th century and several earlier or adjacent periods.
  • 17.83% is the highest average drift reduction, occurring for 20 →13, closely followed by 13 →20 at 17.72%.Comparisons involving the 13th century are particularly sensitive to isolated innovative, stylistically distinct, or domain-specific usages.
  • 29,299 total instances and 5,564 drifted instances occur across permutations targeting the 20th century, the largest corpus-level instance pools reported.Four of the five relevant permutations individually exceed one thousand drifted instances.
  • The lemma මාර්ග\ mɑːrgʌ diverges between a diverse, worldly 20th-century context and a narrower 13th-century vocabulary dominated by meditative and doctrinal terms.The 13th-century contexts are heavily represented by the canonical work විශුද්ධිමාර්ගය\ vɪʒʊððʰi mɑːrgʌjʌ, associated with a 46.27% reduction in structural drift.
  • The honorific වහන්ෙස්\ vʌhʌnseː retains its core function while its surrounding register shifts from monastic and genealogical vocabulary toward modern institutional and epistolary usage.Dense title chains are identified as the primary reason for a 42.67% reduction in drift.
Loading 2609.08609v1…