A Hybrid Protocol for Large-Scale Semantic Dataset Generation in Low-Resource Languages: The Turkish Semantic Relations Corpus

• Existing Turkish semantic resources lack the scale and domain coverage needed by modern neural NLP, while Turkish morphology, inherited projection biases, and costly manual annotation compound resource creation.The introduction notes that agglutinative Turkish requires larger vocabularies for equivalent coverage and that legal, medical, and technical terminology is particularly undercovered.Source: §1 · p. 1
• The hybrid protocol targets computational efficiency and linguistic quality, combining large-scale automation with dictionary-based quality control and offering a method intended to generalize to other low-resource languages.The paper says this generalization requires minimal language-specific adaptation.Source: §1 · pp. 1–2
• The context-preparation phase expands a 77,000-term curated lexicon to 110,000 entries, then uses FastText embeddings and Agglomerative Clustering to form 13,000 semantic clusters.Named Entity Recognition–based augmentation supplies the additional entries; clusters provide context for downstream relationship classification.Source: §1 · p. 1
• Gemini 2.5-Flash induces and labels relationships within clusters for about $65, while dictionary filtering yields 16,000 validated synonym pairs from roughly 20,000 entries.The dictionary filters retain high-precision cases with at most two synonym candidates.Source: §1 · p. 1
• The Turkish Semantic Relations Corpus contains 843,000 annotated pairs across synonym, antonym, and co-hyponym relations, and is described as the largest native Turkish semantic resource to date.The corpus is released in JSONL format.Source: §1 · p. 2
• 90% top-1 retrieval accuracy for synonym pairs and 90% F1-macro for classification were achieved by downstream models trained with the dataset.The introduction presents these results as evidence of the dataset’s utility for semantic systems.Source: §1 · p. 2

2.1 Semantic Resources for Low-Resource Languages

• WordNet’s labor-intensive curation constrained under-resourced coverage; Turkish Tree Bank inherited source organization and potential cultural biases, while KeNet has approximately 80,000 synsets and limited specialized coverage.The existing Turkish resources primarily cover general vocabulary and lack depth in specialized domains.Source: §2.1 · p. 2

2.2 LLM-Augmented Dataset Generation

• Prior work found that GPT-4 generates synonym and antonym pairs with accuracy comparable to crowdsourced annotations (Chen et al., 2023), while LLM paraphrasing improves low-resource task performance; this work combines LLM generation with clustering context and dictionary validation.Prompting can extract world knowledge encoded in multilingual LLMs for dataset generation.Source: §2.2 · p. 2

2.3 Embedding-Based Clustering for Semantic Organization

• FastText’s sub-word modeling provides useful representations for rare forms and novel compounds in morphologically rich languages (Bojanowski et al., 2016), and this work uses Agglomerative clustering as context for LLM classification rather than as its final output.Prior hierarchical-clustering applications include word-sense induction and coreference resolution.Source: §2.3 · p. 2

2.4 Antonym-Synonym Distinction in Vector Spaces

• Synonyms and antonyms can be vector-neighbors because of shared contexts (Mohammad et al., 2013); unlike counter-fitting, which requires lexical constraints (Mrkši´c et al., 2016), supervised synonym/antonym/co-hyponym classification filters antonyms from synonym candidates.Counter-fitting pushes antonyms apart and draws synonyms together, but depends on precompiled lexical resources.Source: §2.4 · p. 3

3.1 Phase I: Context Preparation

• Agglomerative clustering using cosine distance on FastText embeddings grouped approximately 110,000 unique Turkish terms into approximately 13,000 semantic clusters, with a 0.4 threshold selected for broad thematic relevance rather than strict synonymy.Source: §3.1 · p. 3

3.2 Phase II: LLM-Based Semantic Enrichment

• Approximately $65 in Gemini 2.5-Flash processing classified 13,000 clusters into synonym, antonym, and co-hyponym relationships.The model was selected for Turkish multilingual capability, structured outputs, low API cost, and a context window up to 1M tokens.Source: §3.2 · pp. 3–4
• Prompt rules restricted synonyms to exact equivalents, treated near-synonyms as co-hyponyms, and prohibited uncertain classifications.The prompt also allowed abbreviations and expansions as synonyms and required structured JSON output.Source: §3.2 · p. 4
• The LLM phase produced approximately 827,000 labeled semantic pairs after parallel batch processing and output cleanup.Post-processing removed self-synonyms, deduplicated pairs, normalized Unicode, and validated JSON structure.Source: §3.2 · p. 4

3.3 Phase III: Integration and Final Formatting

• Filtering a Turkish synonym dictionary yielded 16,000 high-reliability pairs to serve as validation anchors in the synthetic dataset.The process retained headwords with at most two synonym candidates, excluded ambiguous or context-dependent entries, and removed overlaps with LLM-generated data; the dataset was stored in JSONL format.Source: §3.3 · p. 4

4.1 Distribution Statistics

• With co-hyponyms accounting for about 72% of pairs, the skew mirrors a general natural-language tendency and motivates weighted loss functions to prevent classifier overfitting to that class.The distribution is attributed to broad semantic relatedness being more frequent than strict synonymy or antonymy.Source: §4.1 · pp. 4–5
• 842,946 pairs comprise 71.96% co-hyponyms (606,612), 17.60% synonyms (148,367), and 10.44% antonyms (87,967); 98.10% are LLM-generated and 1.90% dictionary-derived.Table 1 presents the corpus distribution statistics.Source: §4.1 · p. 4, Table 1 · p. 5
• The low Type–Token Ratio (0.02) indicates interconnected pair structure, while a maximum input length of 37 tokens (mean 11.04) supports a 64-token classifier sequence limit.The 64-token upper bound also avoids unnecessary padding overhead.Source: §4.1 · p. 5

4.2 Domain Coverage

• The dataset spans legal, financial, technical, and administrative vocabulary; roughly 4–5% of generated instances contain foreign legal terminology routinely used in Turkish legal practice.Covered areas include contract, criminal, administrative, and constitutional law; banking, insurance, taxation, and corporate finance; IT, engineering, and medicine; and government procedures, institutions, and regulation.Source: §4.2 · p. 5

5.1.1 Data Preparation

• Including co-hyponyms as hard negatives worsened embedding performance compared with training without them.The training data comprised approximately 55,000 unique (query, positive, hard_negatives) triplets; positives were synonyms, while hard negatives included antonyms or co-hyponyms.Source: §5.1.1 · p. 5

5.1.4 Embedding Model Results

• 90% top-1 retrieval accuracy for synonym pairs was achieved by the embedding model on a held-out test set.The measure is the proportion of queries for which the true synonym appears among the top-k retrieved results; multilingual-e5-large (v1) achieved the best retrieval accuracy among seven candidates.Source: §5.1.4 · p. 5, Figure 2 · p. 6

5.2.1 Phase 1: Model Candidate Selection

• Turkish-e5-large was selected from six sentence-embedding candidates after achieving 0.87 F1-macro and stable training dynamics in Phase 1.The six candidates were evaluated under identical conditions, and the selected model was used in subsequent experiments.Source: Table 3 · p. 6, §5.2.1 · p. 6, §5.2.3 · p. 6

5.2.2 Experimental Framework; 5.2.4 Phase 2: Final Optimized Training

• The experiments used two phases: identical-condition benchmarking of six candidates, followed by optimized training of the selected model on upgraded hardware.Final training used NVIDIA L40S hardware, batch size 128, learning rate 3 × 10^-5, and maximum sequence length 64 tokens.Source: §5.2.2 · p. 6, §5.2.4 · p. 6

5.2.5 Classification Model Results

• 90% F1-macro was achieved by the final classification model on the held-out test set.Per-class F1-scores were 0.83 for synonyms, 0.92 for antonyms, and 0.94 for co-hyponyms; macro-averaged precision and recall were 0.88 and 0.92. The weighted loss mitigated class imbalance, with synonyms representing 17.60% of training data.Source: Table 5 · p. 7, §5.2.5 · p. 7

6.1 Strengths of the Hybrid Approach

• The hybrid pairs unlabeled FastText clustering, LLM enrichment for nuanced relations, and curated dictionaries as validation anchors, while its synonym/antonym/co-hyponym scheme distinguishes shared semantic space from synonymy.This distinction is particularly valuable for models that must capture both similarity and specificity.Source: §6.1 · p. 7

6.2 Limitations

• Legal-domain vocabulary may bias coverage against casual or conversational Turkish, while approximately 98% LLM-generated data may propagate LLM-specific biases downstream.Source: §6.2 · p. 7
• The 2025 terminology snapshot needs periodic updates, and Turkish suffixation is not systematically covered across full paradigms, leaving some inflected forms absent.Source: §6.2 · p. 7

6.3 Generalizability

• Cross-linguistic transfer requires FastText embeddings and clustering, a target-language-capable LLM, and a dictionary; the authors estimate $50–100 in LLM API costs for languages with these resources.FastText embeddings are available for 157 languages.Source: §6.3 · p. 7

7 Conclusion

• The corpus contains 843,000 annotated semantic pairs, combining LLM enrichment with dictionary validation, and is described as the largest native Turkish semantic resource to date.It is produced by a scalable hybrid protocol for generating semantic relationship datasets in low-resource languages.Source: §7 · pp. 7–8
• 90% retrieval accuracy for embedding and 90% F1-macro for classification demonstrate the corpus’s practical utility.Source: §7 · p. 8
• Independence from extensive manual annotation makes the protocol generalizable to other low-resource languages, and the dataset and trained models are released to facilitate research.Source: §7 · p. 8

A Representative Dataset Examples

• Examples include sözleşme/mukavele for contract, alıcı/satıcı for buyer/seller, and hukuk/ceza for civil law/criminal law.These illustrate synonymy, financial opposition, and domain-specific co-hyponymy, respectively.Source: A Representative Dataset Examples · p. 9
• Table 6 presents example Turkish semantic pairs for synonyms, antonyms, and co-hyponyms.English translations are provided in parentheses for international readers.Source: Table 6 · p. 9, A Representative Dataset Examples · p. 9

B LLM Semantic Enrichment Prompt Template

• The prompt directs Gemini 2.5-Flash to classify relationships within each cluster as synonyms, antonyms, or co-hyponyms under strict categorization and output-formatting rules.The complete system prompt is provided in Figure 3.Source: B LLM Semantic Enrichment Prompt… · p. 9

C NER-Based Term Augmentation Prompt

• Figure 4’s NER prompt extracts legal-domain concepts, technical terms, time-bound events, and timeless facts, adding 33,000 terms and expanding the lexicon from 77,000 to 110,000 entries.The prompt supports term augmentation from legal documents.Source: C NER-Based Term Augmentation Prompt · p. 9, Figure 4 · p. 11