Source-linked AI summary
Preference Optimization Improves Definition Generation: Semantic Change in EU Institutional Sustainable Finance Disclosures
Mariia Fedorova, Andrey Kutuzov, Gustavo Cadenas Delascio
TL;DR
Definition generators can produce excessive, overly granular definitions for a single lexical sense, complicating semantic-change analysis. The paper applies DPO to encourage consistent definitions and evaluates the approach on an academic benchmark and EU sustainability documents. DPO reduces definitions per target word, but benchmark effects vary by model family, while a Llama DPO model supports tracing sustainability-vocabulary shifts in EIB documents.
Problem
Definition generators can produce slightly different definitions for the same sense, creating noise senses that complicate analysis of semantic change.
Source: §1 · p. 1Method
The authors fine-tune definition generators with DPO, preferring one generated definition per sense over other outputs for that sense.
Source: §1 · p. 2, §5.2 · p. 4Results
Takeaways & Limitations
The EIB analysis makes visible how the Bank’s operational vocabulary is reclassified over time in response to external political and regulatory mandates.
Source: §7 · p. 8Qualitative results trade off granularity and domain sensitivity: DPO improves highly granular Llama outputs but damages OLMo-3 performance and can erase specialized terms.
Source: §6.4 · pp. 7–8
Abstract
from arXiv · showhide
Dictionary-like definition generators are known to be useful for lexical semantic change detection (LSCD). However, they suffer from overly granular sense distribution, producing excessive number of definitions for one and the same lexical sense. In this work, we show that fine-tuning with Direct Preference Optimization (DPO) helps to overcome this problem. We test our approach both quantitatively on a curated academic LSCD benchmark and qualitatively on real-world data for a practical task of auditing diachronic changes in sustainability terminology within project documentation of EU development finance. All models, code and predictions produced in this study are publicly available at https://github.com/ltgoslo/dpo-for-definition-generation.
1 Introduction
Definition generators support interpretable lexical semantic change analysis, but can produce many slightly different definitions for the same sense. The paper uses DPO to encourage consistent definitions and evaluates the approach on benchmarks and EU sustainability documents.
1 Introduction
- Definition generators are supervised-fine-tuned language models trained on target words in context paired with human-produced definitions.The paper illustrates the setup with “bank” used in a river context.Source: §1 · p. 1
- Slightly different definitions for the same sense create “noise senses” that make it harder for domain experts to analyze predictions and interpret change cases.The definitions can differ by only one word or punctuation mark while remaining semantically similar.Source: §1 · p. 1
- Preference learning selects one definition per sense as preferred and labels other model-generated definitions as rejected, encouraging more uniform outputs through DPO.The desired behavior is similar or identical definitions for a target word used in the same sense.Source: §1 · p. 1
- DPO consistently reduces the number of definitions per target word, while its effect on graded semantic change detection varies by base model family.The authors report that benchmark performance can improve or degrade depending on the family.Source: §1 · p. 1
- The paper introduces OLMo-3-7B-based definition generators that outperform prior Llama3-8B-based work.The authors also report improved semantic change detection unless the model is already strong.Source: §1 · p. 2
- The study evaluates DPO on semantic change detection and qualitatively on EU documents, where a domain expert confirms the resulting analysis.The EU-document analysis concerns sustainability terminology.Source: §1 · pp. 1–2
2 Related work
Prior semantic change detection commonly represents word meaning with embeddings, while definition generation offers sense-based, human-readable representations. The related work situates this approach among definition-generation models and language-model methods for sustainability analysis.
2 Related work
- State-of-the-art semantic change approaches use word-in-context embeddings and Average Pairwise Distance, while definition generation represents senses with human-readable explanations.XL-Durel supplies target-word token embeddings for APD.Source: §2 · p. 2
- Earlier definition-generation work used T5 models, followed by decoder-only Llama 3 and multilingual models trained on DBnary data.Prior work also compared encoder-decoder and decoder-only architectures.Source: §2 · p. 2
- Sustainability-disclosure research has used language models to extract climate-risk passages and classify corporate communication into environmental, social, and governance categories.One cited study reports a correlation between disclosure volume and external ESG ratings.Source: §2 · p. 2
3 Datasets for supervised fine-tuning and preference optimisation
The study uses DBnary training and development splits for supervised fine-tuning and preference optimization. Preference pairs are constructed from model outputs so that examples of the same sense share a preferred definition.
3 Datasets for supervised fine-tuning and preference optimisation
- DBnary training and development data are divided into SFT and DPO subsets, with the SFT subset used only for OLMo-3.The splits are stratified by the number of unique gold senses and include single words and multi-word entities.Source: §3 · p. 2
- For each sense, generated responses are paired so that the preferred definition is the one with the shortest Levenshtein distance to the gold definition.The example labels “Government by youth” as preferred over other generated definitions for “juvenocracy.”Source: §3 · pp. 2–3
- Table 1 reports DBnary split statistics before postprocessing the DPO split.The appendix provides examples of SFT and DPO training instances.Source: §3 · pp. 2–3, Table 1 · p. 3
- After postprocessing, only senses with at least two usage examples remain, yielding 20 016 training examples and 2 086 development examples.The counts are reported for the resulting DPO split.Source: §3 · p. 3
4 Domain-specific corpus: EIB Environmental and Social Data Sheets
The domain analysis uses EIB Environmental and Social Data Sheets (ESDS), operational project documents spanning 2014–2026. The corpus provides sentence-level usage examples and a domain-expert-selected sample of noun phrases for qualitative analysis.
4 Domain-specific corpus: EIB Environmental and Social Data Sheets
- ESDS are mandatory appraisal-phase environmental and social due-diligence records submitted to the EIB Board for financing approval.The documents are produced for every EIB-financed project.Source: §4 · p. 3
- The corpus spans 2014 to 2026, establishing a historical baseline of EIB sustainability discourse before capturing the 2021 Global Gateway launch and concurrent repositioning of the EIB as a climate bank.This allows distinction between semantic shifts associated with specific geopolitical mandates and broader baseline drift in sustainability vocabulary.Source: §4 · p. 3
- The ESDS corpus contains 4 201 EIB project documents, of which 4 189 are English, and yields 151 041 sentences for lexical semantic change analysis.The study is limited to English documents after document-level language identification.Source: §4 · p. 3
- A domain expert samples target phrases from noun phrases extracted with flair/chunk-english for the qualitative analysis.The corpus was assembled for analysis of diachronic change in sustainability concepts.Source: §4 · p. 3
5 Experiments
The experiments compare definition-generation models and test whether DPO reduces redundant senses while preserving semantic-change detection performance. They also evaluate a pipeline for comparing changes across time periods.
5.1 Baselines
- Baselines include mT0-xl and Llama3Dictionary trained on a WordNet–Oxford–CoDWoE mixture, plus LlamaDictionary-en_ML38BI trained on DBnary.The EIB website’s terms are unclear about republishing or creating research derivatives, so the authors will publish processing code but not the collection.Source: §5.1 · pp. 3–4
5.2 Our definition generators
- The authors introduce Llama-DPO, OLMo-SFT, and OLMo-DPO, adding an open OLMo model because Llama’s training data are unknown and its license restrictive.OLMo first receives supervised fine-tuning on DBnary before preference optimization.Source: §5.2 · p. 4
5.3 Hyperparameters and hardware
- DPO uses one epoch with LoRA and bf16; overly aggressive β values of 0.4–0.1 increased prediction diversity, so the authors used 0.5.The SFT round for OLMo-3 was allowed up to 20 epochs and stopped after epoch 2.Source: §5.3 · p. 4
5.4 Change detection
- The change-detection pipeline compares sense distributions across SemEval periods with Jensen–Shannon divergence and tunes full-merging threshold 22 on DBnary development data.The tuned threshold optimizes the number of senses per target word before final evaluation on the SemEval test set.Source: §5.4 · p. 4
- Agglomerative clustering groups definition embeddings, while XL-Durel APD reranks targets so experts can inspect which senses were gained or lost.Clustering uses complete linkage and cosine-distance threshold 0.2; the combined views show changed words and their specific sense changes.Source: §5.4 · pp. 4–5
5.5 Results
- SemEval 2020 Task 1 contains only 37 target words, limiting fair rankings of systems with close scores whose confidence intervals overlap.The study evaluates graded semantic change detection on subtask 2 and compares several merging thresholds and clustering strategies.Source: §5.5 · p. 5
- Definition generation does not outperform XL-Durel+APD, and overlapping confidence intervals prevent conclusions about whether OLMo or LlamaDictionary-ML38B is better or DPO is necessary.The best absolute score is achieved without postprocessing; the authors describe the models as passing a sanity check and preferable to previous definition-generation work.Source: §5.5 · p. 5
- DPO reduces predicted senses and moves model predictions toward DBnary’s gold sense distribution, although OLMo-3 models produce over twice DBnary’s share of ambiguous words.The experiments also find that reproducing one dataset’s sense distribution may not translate directly to higher semantic-change detection performance.Source: §5.5 · p. 5
6 Case study: qualitative sustainability analysis
The case study applies definition generation to EIB documents to examine whether sustainability terminology changes in meaning or is strategically ambiguous. It compares SFT and DPO models across architectures, using APD to select terms for qualitative analysis.
6 Case study: qualitative sustainability analysis
- The case study compares SFT and DPO models across Llama and OLMo-3 on EIB documents, focusing on change detection and definition granularity.The analysis is motivated by contested sustainability terminology and the need to distinguish semantic evolution from strategic ambiguity.Source: §6 · pp. 5–6
6.1 Validating Change Detection via APD
- Domain validation placed an empirical APD threshold near 0.27 for substantive policy shifts, while stable terms scored below 0.09 and contested terms scored higher.The shift in “environmental documents” aligns with the 2020 EU Taxonomy, while “new hydrogen” scored 0.391.Source: §6.1 · p. 6
6.2 Sense Granularity and Preference Learning
- DPO reduces LlamaDictionary’s excessive sense granularity, while OLMo-3 DPO reaches the lowest granularity but at severe cost to domain utility.The Llama SFT baseline produced 130 senses for “quality management”; OLMo-3 SFT had the lowest hallucination rate, while OLMo-3 DPO produced the lowest granularity.Source: §6.2 · p. 7
6.3 Tracking Semantic Shifts in Policy Implementation
- The LlamaDictionary-en_ML38BI DPO model was most sensitive to policy-relevant nuance, tracing financialisation, normative stretching, and regulatory absorption in EIB terminology.Examples include “additional CO2 emissions” shifting from a physical to a financial meaning and “ambitious target” acquiring an EU-specific regulatory sense.Source: §6.3 · p. 7
6.4 Observed Limitations in Definition Generation
- OLMo-3 models trade policy sensitivity for lower granularity, and DPO damages OLMo-3’s qualitative performance while helping make Llama’s granular outputs more usable.OLMo-3 definitions can be overly generic and miss institutional context; Llama DPO smooths structural noise while preserving policy sensitivity.Source: §6.4 · pp. 7–8
6.5 Bridging Detection and Interpretation
- Definition-based analysis complements scalar change scores with human-readable evidence that can help researchers distinguish genuine meaning change from strategic expansion of concepts.The paper presents this as a way to examine operational vocabulary in project documents at corpus scale rather than relying on manual document review.Source: §6.5 · p. 8
7 Conclusion
The conclusion argues that DPO may reduce overly granular senses and improve definition-based semantic change detection. It also reports practical findings about merging, training data, and applying the pipeline to EU development-finance documents.
7 Conclusion
- DPO may reduce overly granular senses and improve semantic change detection in the definitions-as-senses paradigm.The authors report that merging definitions is largely unnecessary for modern definition generators.Source: §7 · p. 8
- DBnary-supported models performed better than models trained on smaller professional-dictionary mixtures, while a 7-billion-parameter model performed on par with an 8-billion-parameter model.The study does not establish why the two differently trained models performed similarly.Source: §7 · p. 8
- The authors argue DPO can save human analysts from reading potentially hundreds of extra definitions, despite its computational cost.They state that preference optimization is performed once and that the published models can be used by social scientists.Source: §7 · p. 8
- Applied to the EIB corpus, the pipeline made visible how the Bank’s operational vocabulary is reclassified over time in response to external political and regulatory mandates.The authors identify LlamaDictionary-en_ML38BI DPO as retaining domain-specific nuance that makes these narrative shifts traceable in operational documents.Source: §7 · p. 8
Limitations
The paper identifies limitations in safety checks, clustering experiments, APD interpretation, and evaluation. Its appendix also reports descriptive APD statistics for the ESDS data.
Limitations
- DBnary was not checked for offensive content, and the base models were not checked for generating it.Source: Limitations · p. 8
- Clustering experiments cover only one sentence-embedding model and one algorithm.Source: Limitations · p. 8
- The pipeline does not threshold APD into real change/no-change labels, leaving that decision to a human expert.Source: Limitations · p. 8
- Human-expert impressions do not exactly align with benchmark performance on a 37-word set of basic English vocabulary; Section 6’s qualitative categorisation also used one expert.The authors call for larger domain-specific benchmarks and independent annotators with inter-rater agreement.Source: Limitations · p. 8
A APD descriptive statistics
- Table 5 reports descriptive statistics of XL-Durel APD scores for the ESDS data.Source: A APD descriptive statistics · p. 11, Table 5 · p. 11
B.1 SFT
This appendix example shows an SFT training instance for contextualized definition generation: a usage example is paired with a gold definition.
B.1 SFT
- The example illustrates pairing a contextual usage prompt with a gold definition in an SFT instance.Source: §B.1 · p. 11
- The example SFT instance uses the prompt “If you keep partying like this, your school-work will suffer” to ask for a definition of “suffer.”Source: §B.1 · p. 11
- The gold prediction for “suffer” is “To become worse.”Source: §B.1 · p. 11
B.2 DPO
The section illustrates a DPO training instance built from LlamaDictionary-en_ML38BI SFT predictions, with chosen and rejected definitions for “Tip.” Both usages share the same original gold definition.
B.2 DPO
- The DPO training instance is created from LlamaDictionary-en_ML38BI SFT predictions.Source: §B.2 · p. 11
- For the Dirks usage, the chosen definition specifies informing someone of a clue or secret knowledge, while the rejected version omits that qualification.Source: §B.2 · p. 11
- The chosen and rejected definitions for “Tip” are paired with the same original gold definition across two usages.The chosen definition comes from the Dirks example; the rejected definition comes from a separate example about institutional trading patterns.Source: §B.2 · p. 11