Source-linked AI summary
Bridging Lexical Divergence: LLM-Assisted, Cost-Efficient, Zero-shot Scientific Entity Linking
Md Rasel Khondokar, Qiao Qiao, Farjana Sultana Samia, Nhat Le, Yuepei Li, Qi Li
TL;DR
Scientific entity linking lacks reliable lexical overlap and often lacks expert-labeled data, while existing LLM-based zero-shot methods are costly and do not filter alias drift. Sci-ZSEL selectively generates entity aliases, filters them with ontology information, and uses the retained matches for pseudo-label fine-tuning. Across five benchmarks it improves over non-fine-tuned baselines, with curated-synonym combinations strongest in most settings and especially useful for no-overlap mentions.
Problem
Scientific EL involves low lexical overlap and specialized terminology, while expert-annotated mention-entity data is unavailable in many domains and existing LLM supervision scales per mention without drift filtering.
Method
Sci-ZSEL generates aliases for selected ontology entities, filters semantic drift using ontology structure and similarity, and matches retained aliases to mentions for pseudo-label fine-tuning.
Results
Across five benchmarks, Sci-ZSEL improves over non-fine-tuned baselines, while combining generated aliases with curated synonyms gives the most robust performance in most settings.
Takeaways & Limitations
Sci-ZSEL is most useful for no-overlap mentions, where curated synonyms alone fall short, and the combined approach is strongest in most evaluation settings.
Takeaways & Limitations
The framework depends on the underlying LLM’s domain knowledge and is currently limited to English, while ontology updates require regenerating aliases and refiltering supervision.
Abstract
from arXiv · showhide
Scientific domain entity linking (EL) differs from general domain EL because mentions and entity names often lack lexical overlap. Another challenge is that specialized terminology is used in the scientific domain, which is rarely encountered in models pretrained on general domains. Therefore, models trained on general domains transfer poorly to scientific domains. To address this, in-domain fine-tuning is the natural remedy. However, many scientific domains lack expert-annotated data, motivating the need for a zero-human-annotation approach. Existing zero-shot methods heavily rely on LLMs to generate aliases across entire mention corpora, which incurs substantial computational cost, and those methods provide no mechanism to filter out noise from LLMs. To address these challenges, we propose Sci-ZSEL, a framework that selectively generates entity aliases with an LLM to control computational cost, and applies an ontology-aware filter to remove aliases that semantically drift toward ontology neighbors. Then, filtered aliases are used to construct pseudo-labeled mention-entity pairs for fine-tuning. To enable evaluation of EL under low lexical overlap, we also release a new animal science EL benchmark linked to three livestock trait ontologies, where mentions and entities exhibit substantially lower lexical overlap than in existing benchmarks. Across five benchmarks, Sci-ZSEL outperforms the non-fine-tuned baseline, is most useful on nonoverlapping mentions, and combining it with curated synonyms gives the best performance in most settings.
1 Introduction
Scientific entity linking is difficult because specialized mentions and ontology entities often diverge lexically, while existing zero-shot approaches are costly and lack alias-noise filtering. Sci-ZSEL addresses these issues through selective entity-side alias generation, ontology-aware filtering, pseudo-label construction, and a new animal science benchmark.
- Motivation: Scientific EL involves substantial lexical divergence, specialized terminology, and ontology hierarchy that general-domain models often fail to represent.Examples include “lambing potential” versus “goat fertility” and “hypertension” versus “high blood pressure.”
- Motivation: Fine-tuning is a natural remedy, but expert-annotated mention-entity pairs are expensive and unavailable in many specialized domains.Prior LLM-based synthesis also incurs per-mention token costs and produces corpus-specific pairs.
- Sci-ZSEL: Sci-ZSEL generates aliases for selected entities, matches them to corpus mentions, and filters semantically drifted aliases using ontology structure and similarity.This entity-side strategy avoids issuing one LLM call per corpus mention.
- Benchmark: The paper releases an animal science EL benchmark built from PubMed articles and linked to three livestock trait ontologies.The benchmark is designed to highlight lexical divergence in scientific EL.
- Results: Across five benchmarks, Sci-ZSEL improves over the non-fine-tuned baseline, while combining generated aliases with curated synonyms performs best in most settings.The combined approach is reported as the strongest overall configuration in the contribution summary.
2 Related Work
Related work shows that general and biomedical EL methods often depend on lexical overlap or costly per-mention LLM supervision. Sci-ZSEL instead generates aliases for selected entities and filters drifted aliases before training.
- General and biomedical EL: General-domain EL methods commonly assume substantial lexical overlap between mentions and entity names, an assumption that fails in scientific text.Scientific mentions may use formal, colloquial, abbreviated, or synonymous forms with little surface overlap.
- General and biomedical EL: Biomedical EL methods improve representation and interaction modeling but still address a setting where abbreviations, acronyms, and synonyms create minimal surface overlap.The cited example contrasts “EGFR” with “Epidermal Growth Factor Receptor.”
- LLM-based supervision: Recent LLM-based EL methods generate supervision per mention, so their token cost scales with corpus size and they lack drift controls.These methods may augment mention contexts or act as end-to-end disambiguators.
- LLM-based supervision: Sci-ZSEL reverses the direction of augmentation by generating aliases for a subset of entities and filtering drifted aliases before pairing them with observed mentions.The design combines LLM semantic reach with a compact retriever-reranker pipeline.
3 Animal Science Benchmark
The animal science benchmark targets difficult trait normalization across three livestock ontologies, emphasizing no-overlap mentions and diverse surface forms. Its construction uses an unlabeled training corpus, expert-annotated tests, and dictionary-assisted mention selection with acknowledged sampling limitations.
- Benchmark motivation: Animal science QTL research requires normalizing variable trait mentions across studies, species, and product lines.The variation motivates separate curation in multiple trait ontologies.
- Dataset construction: The benchmark links recognized mentions to CMO, VT, and LPT, uses AnimalQTLdb as the training corpus, and provides no annotated training set.CuPUL recognizes trait mentions, while the test dataset is annotated separately.
- Overlap categories: HO denotes substring overlap, LO partial overlap without substring containment, and NO no lexical overlap between mention and entity name.These categories characterize different levels of lexical divergence.
- Benchmark difficulty: The animal science test sets have higher NO rates and more diverse mentions per entity than existing biomedical benchmarks, while their ontologies contain fewer synonyms.Together, these properties make the new benchmark potentially harder.
- Limitations: Expert review is costly, so the test sets are small and mention selection may bias evaluation toward ontology-anticipated surface forms.Dictionary matching filters the initial pool, although experts also manually collected no-overlap cases.
4 Preliminaries
The paper defines zero-shot EL as linking without human-labeled mention-entity training pairs and describes pretrained retriever and reranker components. Benchmark statistics summarize overlap, mention-entity counts, ontology size, and synonym coverage.
- Task definition: In the zero-shot setting, entities have names, definitions, and possible synonyms, while mentions include extracted text and context but no human-labeled training pairs.The task predicts each mention’s referent entity in the target ontology.
- Retrieval: A bi-encoder embeds mentions and entities into dense vectors, scores them by dot product, and returns the top-K entities.BLINK is used as the pretrained bi-encoder backbone.
- Benchmark statistics: Table 1 reports test-set samples, HO/LO/NO percentages, unique mentions and entities, ontology entities and synonyms, average synonyms per entity, and synonym coverage.QTLCMO, QTLVT, and QTLLPT are test-only datasets.
- Reranking: A cross-encoder jointly encodes a mention, its context, and candidate entities to produce reranking scores.The evaluated cross-encoder backbones are BLINK and ReS.
5 Methodology
Sci-ZSEL reduces alias-generation cost by selecting relevant ontology entities, generating aliases from their definitions, filtering semantic drift, and using matched aliases to create pseudo-labeled pairs for retriever and reranker fine-tuning.
- Entity Selection: Sci-ZSEL selects a subset of ontology entities from the unlabeled corpus, making alias supervision reusable across similar-domain corpora.Selection targets entities whose names appear in mentions and entities identified by the pretrained bi-encoder as top candidates.
- Alias Generation: The framework prompts an LLM with selected entity definitions to generate aliases that bridge corpus mentions and ontology entities.Aliases are generated for all selected entities, producing an alias set for pseudo-label construction.
- Ontology-Aware Filtering: Ontology-aware filtering removes aliases closer to a parent, child, or sibling than the target entity, or below the target-name similarity threshold τ = 0.9.Similarity is computed with BioLORD embeddings; aliases are retained only when they are sufficiently similar to the target and not drifting toward a neighbor.
- Pseudo-Label Construction: Filtered aliases are matched exactly against normalized corpus mentions to form pseudo-labeled mention-entity pairs.The construction uses whitespace and case normalization before testing whether a mention matches a filtered alias.
- Pseudo-Label Construction: Curated ontology synonyms provide an additional source of pseudo-labeled pairs whenever a synonym exactly matches an observed corpus mention.The method treats expert-curated synonyms as reliable lexical bridges.
- Fine-Tuning: The retriever uses ontology-aware sampling, while the reranker uses standard candidate-set fine-tuning to rank pseudo-labeled entities highest.Retriever strategies vary whether ontology neighbors are negatives or promoted positives; the reranker treats the pseudo-labeled entity as positive against sampled candidates.
6 Experimental Studies
Across five benchmarks, Sci-ZSEL improves scientific entity-linking performance over non-fine-tuned baselines, especially when lexical overlap is low, while combining generated aliases with curated synonyms is usually strongest.
- Evaluation Setup: Sci-ZSEL is evaluated on five benchmarks spanning synonym-rich biomedical and synonym-sparse animal science datasets.The benchmarks include NCBI Disease, BC5CDR, QTLCMO, QTLVT, and QTLLPT.
- Alias Generation: Sci-ZSEL reduces LLM cost by generating aliases for selected entities rather than every corpus mention, while using fewer calls on large ontologies and fewer input tokens per call across all benchmarks.Selected entities cover under 0.6% of MeSH and 6.0% of MEDIC, compared with higher proportions for the smaller animal-science ontologies.
- Alias Generation: Over 80% of aliases generated for synonym-sparse animal-science ontologies become new lexical bridges rather than matching curated synonyms or entity names.This result is reported for the bi-encoder top-1 setting on CMO, VT, and LPT.
- Pseudo-Pair Quality: 90.32% for NCBI and 91.40% for BC5CDR: ontology-aware filtering raises pseudo-pair accuracy from noisy raw alias generation to over 90%.Raw LLM alias generation yields 72% pseudo-label accuracy across the two datasets before filtering.
- Retriever Results: +13.35%, +15.36%, and +23.96%: retriever fine-tuning improves Recall@64 on QTLCMO, QTLVT, and QTLLPT, respectively, versus non-fine-tuned backbones.The corresponding gains on synonym-rich NCBI and BC5CDR are smaller: +2.68% and +0.35%.
- Reranker Results: Sci-ZSEL plus curated synonyms achieves the highest BLINK reranker Recall@1 on all five benchmarks and dominates MRR on four, while ReS results are more dataset-dependent.For ReS, the synonym setting leads on NCBI and QTLCMO, whereas the combined approach leads on BC5CDR, QTLVT, and QTLLPT.
- Reranker Results: +49.12 percentage points: Sci-ZSEL improves the NO category over non-fine-tuned BLINK and ReS backbones, where curated lexical coverage is limited.The paper reports that improvements in these highly challenging cases frequently double or triple baseline performance.
- Ablations: Removing ontology-aware filtering can push reranker accuracy below the non-fine-tuned baseline, especially on animal-science benchmarks.Ablations also show PC-POS achieves the best retriever Recall@64 on four of five benchmarks, while Sci-ZSEL with synonyms is the strongest pseudo-pair source on four of five.
8 Conclusion
Sci-ZSEL addresses severe lexical divergence in scientific entity linking through cost-aware alias generation and ontology-aware filtering. Across five datasets, it improves over non-fine-tuned baselines, especially for lexically divergent mentions, while curated synonyms strengthen robustness.
- Sci-ZSEL bounds LLM alias generation to selected entities and uses an ontology-aware filter to discard drifted aliases.The filtered aliases support pseudo-labeled training pairs for zero-shot domain adaptation.
- Across five datasets, Sci-ZSEL significantly outperforms non-fine-tuned baselines, with absolute gains in the lexically divergent domain.
- Combining Sci-ZSEL with curated synonyms yields the most robust reranker configuration, mainly because no-overlap mentions benefit when curated synonyms alone fall short.
- Ablation studies confirm that the ontology-aware filter is essential to prevent LLM noise from degrading accuracy below baseline levels.
9 Limitations
Sci-ZSEL’s limitations concern dependence on LLM domain knowledge, English-centric components, ontology snapshots, and excluded NIL entities. These constraints affect reproducibility, multilingual extension, update handling, and practical coverage.
- Sci-ZSEL depends heavily on the underlying LLM’s domain knowledge, so weaker biomedical or animal-science coverage can produce noisier aliases and increase filtering burden.
- The framework and evaluation are strictly English-centric, requiring multilingual LLMs and domain-tailored semantic-similarity models for other languages.
- Closed-source LLM adaptations may face reproducibility risks if silent updates alter alias distributions.
- Ontology updates require alias regeneration and rerunning the filtering process because pseudo pairs and filtering decisions are tied to fixed ontology snapshots.
- Sci-ZSEL does not handle NIL entities, which were excluded from evaluation despite many benchmark mentions lacking referents in all three ontologies.
10 Ethical considerations
The animal science benchmark was developed through graduate-student annotation, domain-expert validation, and disagreement resolution using publicly available PubMed articles. The alias-generation LLM processed public ontology definitions rather than private or human-generated data.
- Four graduate-student linking annotators and one domain-expert curator developed and validated the animal science benchmark.The curator resolved disagreements and performed annotation validation.
- Benchmark source texts came entirely from publicly available PubMed articles and contained no personally identifiable information.
- Llama 3.2 3B Instruct generated entity aliases using definitions from public ontologies without processing human-generated or private data.
- BLINK, ReS, and BioLORD were open-source components deployed under their published usage licenses.
A.1 Entity Generation Prompt
The entity-generation prompt uses one template across five datasets, varying the domain string and three ontology-derived in-context examples. It asks an LLM to generate scientifically accurate names from entity definitions and return them as comma-separated names.
- A single prompt template is used across all five datasets, varying only the domain string and three in-context examples drawn from the target ontology.
- The system message specifies domain expertise, while the user message supplies an entity definition to be named.
- The prompt instructs the model to generate a scientifically accurate name that reflects the definition and aligns with standard terminology.
- Three ontology-specific in-context examples provide definition–generated-name demonstrations before the target definition.
- The output format requires all possible names for the definition, separated by commas, with no additional text.
A.2 Number of Training Samples
Training-sample counts vary substantially across datasets and settings because curated-synonym density and alias-filter discard rates differ. The resulting data support fine-tuning and the reported ablations, including ontology-aware sampling and construction strategies.
- A.2 Number of Training Samples: Sample counts vary by an order of magnitude across dataset–setting cells.The table reports training samples used for retriever and reranker fine-tuning.
- A.2 Number of Training Samples: Curated synonyms yield 3,283 NCBI, 4,676 BC5CDR, 871 QTLCMO, 1,854 QTLVT, and 477 QTLLPT pairs.These counts track synonym/entity ratios in the ontology statistics.
- A.2 Number of Training Samples: Sci-ZSEL retains 74% of NCBI, 77% of BC5CDR, 64% of QTLCMO, 29% of QTLVT, and 22% of QTLLPT unfiltered pairs.The filter discards a larger fraction of generated aliases for the animal science ontologies.
- A.2 Number of Training Samples: Sci-ZSEL + Synonym is the largest training set across all benchmarks.It combines generated aliases with curated synonyms.
- A.7 Detailed Ablation Analysis: Ontology-aware sampling outperforms vanilla negative construction, with PC-POS usually strongest and RM-PCS best on QTLVT.PC-POS achieves the highest Recall@64 on NCBI, BC5CDR, QTLCMO, and QTLLPT, while RM-PCS reaches 88.69% on QTLVT.
- A.7 Detailed Ablation Analysis: Removing the ontology-aware filter can reduce reranker accuracy below the non-fine-tuned baseline, with a largest overall Recall@1 drop of 7.50% on QTLCMO using ReS.Drifted aliases can train links toward neighboring entities rather than the true referent.