Source-linked AI summary
Beyond Cosine Similarity: Taming Semantic Drift and Antonym Intrusion in a 15-Million Node Turkish Synonym Graph
Ebubekir Tosun, Mehmet Emin Buldur, Özay Ezerceli, Mahmoud ElHussieni
TL;DR
Distributional embeddings confuse synonyms with antonyms and other related terms, while large Turkish synonym resources remain sparse and graph clustering can suffer from drift and polysemy. The paper combines disambiguated supervision with topology-aware soft-to-hard clustering, producing 2,905,071 final clusters from 15 million terms and 520 million verified synonym links.
Problem
Distributional embeddings place synonyms and antonyms in proximate regions, while Turkish synonym resources are sparse and existing clustering methods struggle with semantic drift and polysemy.
Method
The method combines an 843,000-pair disambiguated dataset, a three-way relation discriminator, symmetry filtering, and topology-aware soft-to-hard clustering.
Results
2,905,071 final clusters are produced from 15 million terms and 520 million verified synonym links, with median size 3, mean size 4.58, and maximum size 86.
Takeaways & Limitations
The resulting resource supports high-precision semantic search and retrieval-augmented generation for morphologically rich languages with sparse synonym databases.
Abstract
from arXiv · showhide
Neural embeddings have a notorious blind spot: they can't reliably tell synonyms apart from antonyms. Consequently, increasing similarity thresholds often fails to prevent opposites from being grouped together. We've built a large-scale semantic clustering system specifically designed to tackle this problem head on. Our pipeline chews through 15 million lexical items, evaluates a massive 520 million potential relationships, and ultimately generates 2.9 million high-precision semantic clusters. The system makes three primary contributions. First, we introduce a labeled dataset of 843,000 concept pairs spanning synonymy, antonymy, and co-hyponymy, constructed via Gemini 2.5-Flash LLM augmentation and verified using human-curated dictionary resources. Second, we propose a specialized three-way semantic relation discriminator that achieves 90% macro-F1, enabling robust disambiguation beyond raw embedding similarity. Third, we introduce a novel soft-to-hard clustering algorithm that mitigates semantic drift preventing erroneous transitive chains (e.g., hot -> spicy -> pain -> depression) while simultaneously resolving polysemy. Our approach employs a topology-aware two-stage expansion-pruning procedure with topological voting, ensuring that each term is assigned to exactly one semantically coherent cluster. The resulting resource enables high-precision semantic search and retrieval-augmented generation, particularly for morphologically rich and low-resource languages where existing synonym databases remain sparse.
1 Introduction
The paper addresses antonym intrusion, semantic drift, and polysemy in large-scale Turkish synonym graphs by combining supervised relation disambiguation with topology-aware clustering. It introduces a large labeled dataset, a three-way discriminator, and a soft-to-hard procedure for producing compact synonym clusters.
- Motivation: The approach targets Turkish resource scarcity, where cosine similarity can group antonyms or broadly related terms and existing resources have limited coverage.The motivation is amplified by Turkish’s morphologically rich, low-resource setting.
- Contributions: 843,000 labeled pairs cover Synonym, Antonym, and Co-hyponym relations through LLM-augmented synthesis and dictionary extraction.The dataset combines 827,000 synthetic instances with 16,000 curated dictionary entries.
- Contributions: 90% F1-macro is achieved by a three-way discriminator that filters antonyms and co-hyponyms beyond raw embedding similarity.Only Synonym predictions above confidence 0.70 pass the semantic gate.
- Contributions: Soft expansion followed by topological voting permits temporary polysemous membership before assigning each term to one best cluster.The reduction stage uses majority rule, specificity preference, and deterministic tie-breaking.
- Scale and output: 15 million terms yield 520 million verified synonym links and 2,905,071 final clusters after symmetry checks and semantic filtering.The resulting graph favors high-confidence synonym relations while reducing transitive-chain errors and handling polysemy.
2 Related Work
Related work shows that distributional representations capture contextual proximity but conflate synonymy with antonymy and other relations. Existing graph methods also struggle with semantic drift, polysemy, and the resource constraints of morphologically rich languages, motivating this paper’s integrated approach.
- Embedding-based semantics: Distributional and transformer embeddings can place synonyms and antonyms near each other because both occur in similar contexts.Similarity therefore does not reliably encode semantic equivalence or relational polarity.
- Lexical constraints: Counter-fitting separates known synonyms and antonyms, but its effectiveness depends on large high-quality lexical constraints.That dependency is especially limiting where manually curated resources are sparse.
- Graph clustering: Standard community detection is vulnerable to semantic drift and hard partitions that merge unrelated senses or discard legitimate polysemous links.Sense-aware approaches mitigate some of these issues by inducing senses before clustering.
- Paper positioning: The paper combines explicit relation discrimination, large-scale supervision, and topology-aware clustering instead of globally reshaping embeddings or relying on fixed lexical constraints.Its soft-to-hard strategy temporarily permits overlap before deterministic voting resolves ambiguity.
- Target setting: The proposed methodology is designed for morphologically rich, low-resource languages and builds high-precision synonym graphs without exhaustive manual lexicons.The stated target setting includes Turkish and other languages with sparse synonym resources.
3 Methodology
The methodology combines multilingual embedding training, LLM- and dictionary-based supervision, scalable similarity retrieval, three-way relation classification, and graph filtering. A two-stage expansion-pruning algorithm then resolves polysemy and limits semantic drift at 15-million-node scale.
- 3.1 Phase 1: Embedding Model Training and Initial Data Generation: Contrastive fine-tuning specializes multilingual-e5-large embeddings for Turkish synonym pairs using CachedMultipleNegativesRankingLoss with τ = 0.07.The resulting embeddings are optimized for synonym pairs rather than generic semantic similarity.
- 3.1 Phase 1: Embedding Model Training and Initial Data Generation: 843,000 labeled pairs are generated from 827,000 LLM-synthesized and 16,000 dictionary-filtered examples.Gemini 2.5-Flash labels clustered Turkish terms as Synonym, Antonym, or Co-hyponym.
- 3.2 Large-scale retrieval: 15 million 1024-dimensional vectors are compressed from roughly 60 GB to about 15 GB using 8-bit scalar quantization.FAISS IVF indexing partitions the search space into 16,384 Voronoi cells for scalable retrieval.
- 3.3 Phase 3: Semantic Relationship Classification: A three-way sentence-pair classifier predicts Synonym, Antonym, or Co-hyponym using the [CLS] representation and weighted CrossEntropy loss.Class weights address imbalance, while pairs classified as Antonym or Co-hyponym are discarded.
- 3.4 Phase 4: Filtering and Conflict Resolution: Symmetry validation removes directional conflicts, reducing 1.3 billion candidates to approximately 520 million verified edges.A pair is removed when one direction predicts Synonym and the reverse predicts Antonym.
- 3.5 Phase 5: Novel Clustering Algorithm for Semantic Drift and Polysemy Resolution: Soft expansion admits terms into multiple clusters using overlap thresholds, while topological voting resolves polysemy into a single best cluster.High-confidence seeding and intersection-ratio thresholds help prevent weak transitive chains.
- 3.5 Phase 5: Novel Clustering Algorithm for Semantic Drift and Polysemy Resolution: 2,905,071.454 semantic clusters are produced with median size 3, mean size 4.58, and maximum size 86.The output is represented in JSON as parent-child equivalence relationships.
4 Experiments & Results
Experiments compare transformer-based classifiers under controlled training conditions and report strong overall and per-class performance for the selected Turkish model.
- Model selection: Six transformer-based models were evaluated under identical conditions for Phase 1 candidate selection.The comparison used the same experimental setup across candidate models.
- Final performance: 90% overall F1-Macro was achieved by the final model.Validation monitoring and best-checkpoint loading were used during training.
- Final performance: 92% F1 on antonym detection and 83% F1 on synonym identification were obtained by the final model.These are reported as per-class results for the final classifier.
4.4 Clustering Results
The clustering pipeline produces compact synonym clusters and uses parent selection to standardize noisy, abbreviated, and formal terminology.
- Clustering scale: 15 million unique terms yielded 2,905,071 final semantic clusters after 520 million synonym pairs were confirmed.Cluster sizes had a median of 3, mean of 4.58, and maximum of 86.
- Clustering scale: Cluster-size statistics indicate broad lexical coverage with limited semantic drift.The reported distribution is median 3, mean 4.58, and maximum 86.
- Parent selection: The parent-selection analysis reports robust handling of complex linguistic phenomena.Examples include OCR correction, formal terminology priority, and institutional-name standardization.
- Parent selection: The system selects “Sosyal Güvenlik Kurumu” as the parent over abbreviations and older institutional names.This example favors the current official name.
5 Discussion and Limitations
The discussion attributes performance to reliable LLM-generated labels, Turkish-specialized pretraining, and topology-aware clustering for difficult semantic distinctions.
- Findings: 90% F1-macro from classifiers trained on Gemini 2.5-Flash labels indicates sufficient quality for downstream applications.The authors present this as support for synthetic data generation over manual annotation.
- Findings: turkish-e5-large outperforms multilingual-e5-large despite identical architecture, supporting the value of Turkish-specific pretraining.The comparison is reported for Turkish semantic relation classification.
- Findings: The two-stage clustering algorithm handles polysemous terms that traditional hard-partitioning algorithms may misclassify.Topological voting provides disambiguation while maintaining computational tractability.
- Class performance: 94.8% F1 on co-hyponyms is associated with class abundance, while the 83.1% synonym F1 reflects the challenge of separating semantically proximate classes.The discussion emphasizes distinguishing synonymy from co-hyponymy, not only synonymy from antonymy.
- Class performance: 90% macro F1 demonstrates separation of “same meaning” from “same category” for antonym-free synonym graph construction.The stated goal is avoiding false merging of related but non-synonymous terms.
- Training dynamics: Figure 2 depicts training and validation loss convergence alongside rising F1-Macro during classifier training.The turkish-e5-large curve reaches a peak score of 0.90 faster than competing baselines.
6 Conclusion
The paper combines large-scale supervision, semantic disambiguation, and topology-aware clustering to build a large Turkish synonym resource with practical retrieval applications.
- Contributions: The system addresses distributional similarity’s conflation of synonymy with other semantic relationships.The conclusion presents this as the central challenge motivating the graph-construction approach.
- Contributions: The three contributions produce 2.9 million synonym clusters from 15 million Turkish terms.They comprise a disambiguated dataset, a 90% F1-macro discriminator, and a soft-to-hard clustering algorithm.
- Implications: The resulting resource enables high-precision semantic search and retrieval-augmented generation for morphologically rich languages with sparse synonym databases.The methodology is designed for cross-linguistic transfer using FastText embeddings, LLM API access, and basic dictionary resources.
- Future work: Future work includes additional morphologically rich languages, morphological paradigm expansion, and dynamic updates for evolving terminology.These directions are stated as extensions of the current approach.
A Additional Examples
The clustering procedure prevents semantic drift from chaining locally valid links into incoherent clusters and uses topological voting to resolve polysemous terms.
- Semantic Drift Prevention: Confidence-based filtering and intersection-ratio thresholds prevent spurious synonym chains that could conflate temperature with mental health.Without filtering, the chain from “Sıcak” (Hot) through “Acı” (Spicy), “Acı” (Pain), “Üzüntü” (Sadness), and “Depresyon” (Depression) would merge unrelated meanings.
- Polysemy Handling: Topological voting assigns the polysemous term “yüz” to the anatomical cluster because it has higher synonym overlap.The term is initially soft-assigned to both anatomical and numerical clusters; related terms capture the numerical sense.
B Detailed Training Metrics
Co-hyponym classification converges quickly during training, while synonym detection varies more across models and stabilizes lower for the best model.
- 0.94 F1: co-hyponym classification converges quickly during training.
- 0.83 F1: synonym detection stabilizes for the best model after greater variation across models.
C Algorithm Pseudocode
The soft-to-hard clustering algorithm processes synonym pairs in descending confidence order, forms or extends candidate clusters, and assigns multiply associated terms through topological voting.
- The algorithm requires synonym pairs with confidence scores and produces final hard clusters.
- Pairs are sorted by descending confidence before the membership map is initialized and pairs are processed.
- For each pair, the algorithm finds candidate clusters for both terms and creates a new cluster when both are unassigned.
- When only one term is unassigned, the algorithm adds it to an existing cluster; an intersection ratio above 0.51 is used in the merge condition.
- Terms belonging to multiple clusters are resolved by topological voting, followed by specificity and deterministic tie-breaking before final assignment.