PaperlayerView exact revision on arXiv

Source-linked AI summary

OntologyAligner: Ontology-Aligned Retrieval and Hierarchy-Guided Large Language Model Reranking for Biomedical Ontology Normalization

Jie Song, Zhichuan Xu, Ziyu Lu, Meng Xiao, Cheng Bi, Yuxin Zhang, Xin Zheng, Xiaoran Li, Qiongfang Cao, Hao Yang, Bairong Shen

arXiv:2609.10055v1cs.AIcs.CL

TL;DR

Biomedical ontology normalization must handle diverse lexical forms and subtle distinctions among related concepts. OntologyAligner combines ontology-aligned retrieval, language-model reranking, and hierarchy-guided refinement, showing supported improvements and stability across configurations.

  • Problem

    Biomedical ontology normalization must resolve diverse lexical variants and subtle distinctions among hierarchically related concepts.

  • Method

    OntologyAligner integrates ontology-aligned retrieval, large language model candidate reranking, and selective hierarchy-guided refinement.

  • Results

    The framework showed statistically supported improvement from hierarchy-guided refinement and stability across multiple implementation configurations.

  • Takeaways & Limitations

    The workflow can support phenotype annotation, cohort construction, literature curation, and biomedical knowledge integration.

  • Takeaways & Limitations

    Evaluation focused on pre-extracted, isolated biomedical phrases, primarily measuring phrase-level ontology normalization performance.

Abstract

from arXiv · show

Biomedical ontology normalization maps free-text expressions to standardized concepts, enabling consistent integration and analysis of biomedical data. This task remains challenging because lexical variation and subtle distinctions among hierarchically related concepts can obscure concept boundaries. We present OntologyAligner, a three-stage framework that combines ontology-aligned retrieval, large language model candidate reranking, and selective hierarchy-guided refinement. We also construct PhenoNormBench, a unified benchmark comprising 13,390 samples from seven Human Phenotype Ontology datasets. OntologyAligner achieved state-of-the-art performance on HPO normalization, with 88.78% Macro Top-1 Accuracy and 86.75% Micro Top-1 Accuracy, exceeding the strongest baseline by 4.85 and 5.07 percentage points, respectively. Ablation analyses showed complementary contributions from all three stages, and sensitivity analyses demonstrated stability across candidate-set sizes and model backbones. Applications to MONDO, MEDIC, and NCBITaxon further established portability to other ontologies. OntologyAligner offers a generalizable framework for accurate mapping of biomedical text to structured ontology concepts. PhenoNormBench and the code are publicly available at https://github.com/zhelishisongjie/OntologyAligner.

INTRODUCTION

Ontology normalization is challenged by lexical variation and subtle distinctions among hierarchically related biomedical concepts. OntologyAligner addresses these challenges through ontology-aligned retrieval, LLM candidate reranking, and selective hierarchy-guided refinement, evaluated with the unified PhenoNormBench benchmark and across multiple ontologies.

  • Motivation: Biomedical ontology normalization maps diverse free-text expressions to stable concepts, supporting data integration, interoperability, clinical decision support, and computational analysis.
  • Challenges: HPO normalization is difficult because lexical variants may have minimal synonym overlap, while parent, child, and sibling concepts differ subtly in characteristics or semantic granularity.
  • Proposed framework: OntologyAligner combines ontology-aligned retrieval, LLM candidate reranking, and selective hierarchy-guided refinement to improve retrieval, semantic discrimination, and local conflict resolution.OAR learns an ontology-specific representation; LCR jointly considers labels, synonyms, and definitions; HGR uses explicit parent-child or sibling evidence when Top-1 predictions differ.
  • Benchmark: PhenoNormBench integrates seven HPO datasets from diverse sources into a unified evaluation benchmark and protocol covering ontology versions and acceptable ID normalization.
  • Contributions and evaluation: OntologyAligner uses concept-level multi-positive InfoNCE for ontology-specific retrieval and selective local hierarchy evidence for distinguishing semantically similar candidates.The framework’s robustness, state-of-the-art performance, and portability were evaluated on MONDO, MEDIC, and NCBITaxon.

OntologyAligner improves HPO normalization in PhenoNormBench

OntologyAligner achieved strong HPO normalization performance across seven PhenoNormBench datasets, outperforming the strongest baseline by 4.85 and 5.07 percentage points in Macro and Micro Top-1 Accuracy. Performance was highest for shorter, lexically overlapping phrases, with a nonmonotonic association with ontology depth.

  • Overall performance: 88.78% Macro Top-1 Accuracy and 86.75% Micro Top-1 Accuracy were achieved across seven datasets.Macro Accuracy was equal-weighted, while Micro Accuracy was pooled over 13,390 samples.
  • Baseline comparison: 4.85 and 5.07 percentage points improved Macro and Micro Accuracy, respectively, over RAG-HPO-TE3L.RAG-HPO-TE3L achieved 83.93% Macro Accuracy and 81.68% Micro Accuracy; all five overall baseline comparisons were significant (P_adj < 0.001).
  • Difficulty-stratified performance: 91.94% Top-1 Accuracy for one-token phrases declined to 73.16% for phrases containing at least four tokens.Two-token and three-token phrases achieved 88.72% and 88.39%, respectively; the latter three groups were significantly below the one-token group.
  • Difficulty-stratified performance: 58.13% Top-1 Accuracy for zero lexical overlap increased to 99.92% for exact overlap.Low- and high-overlap groups achieved 74.58% and 84.06%, and all nonexact groups differed significantly from exact overlap (all P_adj < 0.001).
  • Difficulty-stratified performance: 84.59% Top-1 Accuracy in the depth 7 to 8 group was significantly below the depth 1 to 4 reference group.Differences for depths 5 to 6 and 9 to 12 were not statistically significant, indicating a nonmonotonic association with ontology depth.

Component contributions and robustness analyses

Ablation results showed complementary gains from ontology-aligned retrieval, LLM candidate reranking, and hierarchy-guided refinement. Performance remained robust across candidate-set sizes and backbone choices, with clear advantages for stronger embedding and LLM backbones.

  • Component ablation: 88.78%/86.75% Macro/Micro Accuracy was achieved by the full OAR + LCR + HGR model, outperforming Raw Retrieval and intermediate variants.Raw Retrieval, OAR, OAR + LCR, and the full model achieved 83.07%/80.82%, 84.71%/82.58%, 88.56%/86.65%, and 88.78%/86.75%, respectively; OAR and LCR improvements were significant at P_adj < 0.001.
  • Component ablation: 0.22 and 0.10 absolute points were added by HGR to Macro and Micro Accuracy, respectively, while removing OAR reduced them by 0.88 and 0.78.Raw Retrieval + LCR + HGR (w/o OAR) achieved 87.90%/85.97%, significantly below the full model (P_adj < 0.001).
  • Candidate-set-size sensitivity: 87.48% to 87.95% accuracy was observed for candidate sets with k ≥ 3, compared with 83.48% at k = 1.Using k = 20 as reference, k = 1 was significantly lower (P_adj < 0.001), whereas k = 3, 5, and 10 showed no significant differences.
  • Embedding-backbone sensitivity: 87.67% accuracy was achieved by TE3L, while TE3-small and BioLORD reached 87.10% without significant differences from TE3L.BioBERT, PubMedBERT, and ClinicalBERT achieved 82.43%, 82.05%, and 81.57%, respectively, all significantly below TE3L (P_adj < 0.001).

Generalization to other biomedical ontologies · DISCUSSION

OntologyAligner generalized across MONDO, MEDIC, and NCBITaxon, achieving Top-1 Accuracy above 92% on all three tasks and outperforming reported comparison methods on MEDIC and NCBITaxon. In the broader evaluation, its modular stages improved HPO normalization, addressed difficult lexical and compositional cases, remained stable across configurations, and supported auditable biomedical annotation workflows.

  • Generalization to other biomedical ontologies: OntologyAligner achieved Top-1 Accuracy values of 97.66%, 92.19%, and 94.13% on FGDD Disease→MONDO, NCBI Disease Corpus→MEDIC, and S800→NCBITaxon, respectively.Each task used a target-ontology-specific projection and dedicated vector index.
  • Generalization to other biomedical ontologies: 92.19% Top-1 Accuracy on MEDIC exceeded BELHD’s 87.60% by an absolute increase of 4.59.Reported comparison methods ranged from 79.90% to 87.60%.
  • Generalization to other biomedical ontologies: 94.13% Top-1 Accuracy on NCBITaxon exceeded GenBioEL+HD’s 89.96% by an absolute increase of 4.17.Reported comparison methods ranged from 78.62% to 89.96%.
  • DISCUSSION: 88.78% Macro Top-1 Accuracy and 86.75% Micro Top-1 Accuracy across 13,390 samples from seven HPO datasets improved over the strongest baseline by 4.85 and 5.07 percentage points, respectively.These results establish the main HPO normalization performance reported in the discussion.
  • DISCUSSION: OAR increased Micro Accuracy from 80.82% to 82.58%, while LCR further increased Micro Accuracy to 86.65% and contributed the largest improvement.HGR provided a smaller yet statistically supported improvement through selective correction of local hierarchical conflicts.
  • DISCUSSION: Accuracy was 58.13% for phrases with zero lexical overlap and 73.16% for phrases containing four or more tokens, identifying paraphrastic and compositionally complex descriptions as challenging.Phrases with exact lexical matches achieved near-ceiling performance, while multiple clinical modifiers remained difficult.
  • DISCUSSION: Candidate sets containing three or more concepts achieved accuracy comparable to 20 candidates, and multiple embedding and LLM backbones produced broadly consistent results.The workflow supports selecting models and candidate-set sizes according to accuracy, cost, latency, and data-governance requirements.
  • DISCUSSION: The modular workflow separates candidate retrieval, semantic ranking, and hierarchical refinement, enabling traceable error analysis, quality control, and human review for biomedical annotation and knowledge integration.Applications include phenotype annotation, cohort construction, literature curation, and biomedical knowledge integration.

Limitations

The study’s evaluation is limited to isolated phrase-level normalization, while broader clinical-pipeline validation remains necessary. Additional limitations concern incomplete use of ontology structure and the predominantly English experimental coverage.

  • Evaluation scope: Evaluation focused on pre-extracted, isolated biomedical phrases, so complete clinical text-processing pipelines require end-to-end validation.The findings primarily measure the ontology normalization module at the phrase level and apply mainly to standard phrase-level evaluation settings.
  • Ontology structure: HGR currently uses parent-child and sibling relationships, leaving part_of, has_modifier, and longer hierarchical paths as potential additional evidence for concept discrimination.Its local conflict resolution primarily distinguishes semantically similar concepts by specificity, semantic scope, and hierarchical position.
  • Language coverage: Experiments primarily covered English biomedical expressions, so applicability to other languages requires further evaluation.Cross-lingual normalization is influenced by differences in medical terminology, word order, abbreviation patterns, modifier structures, translation quality, and ontology quality.

Future work · Conclusions

OntologyAligner combines ontology-aligned retrieval, LLM candidate reranking, and hierarchy-guided refinement for biomedical ontology normalization, achieving strong HPO benchmark performance and portability across target ontologies. Future work focuses on full clinical pipelines, richer ontology relations, and multilingual evaluation.

  • Future work: Future work should extend OntologyAligner into a complete clinical text-processing pipeline jointly performing entity recognition, contextual understanding, and ontology normalization.This would broaden the current normalization framework toward integrated clinical text processing.
  • Future work: Future studies should expand HGR beyond parent–child and sibling relationships by investigating part_of, has_modifier, and longer hierarchical paths.Ontology-specific evidence organization and reasoning rules for different relation types may improve discrimination of complex concept boundaries.
  • Future work: Multilingual biomedical normalization should integrate multilingual terminology resources, ontology localization mappings, and manually annotated corpora.A unified benchmark spanning languages, region-specific terminology, and cross-lingual expressions would enable systematic evaluation of multilingual transfer performance.
  • Conclusions: OntologyAligner integrates ontology-aligned retrieval, LLM-based candidate reranking, and selective hierarchy-guided refinement in a three-stage normalization framework.OAR aligns queries with target-ontology terminology, LCR jointly considers labels, synonyms, and definitions, and HGR uses local parent–child and sibling relationships.
  • Conclusions: 88.78% Macro Top-1 Accuracy and 86.75% Micro Top-1 Accuracy were achieved on PhenoNormBench’s 13,390 samples.These results represent absolute improvements of 4.85 and 5.07 percentage points, respectively, over the strongest baseline.
  • Conclusions: OAR, LCR, and HGR each made independent contributions to overall performance, with LCR providing the largest improvement.HGR supplied a statistically supported additional correction for a small number of local hierarchical ambiguities.
  • Conclusions: Experiments on MONDO, MEDIC, and NCBITaxon showed that the workflow generalized to phenotype, disease, and biological-taxon normalization.Application required training an ontology-specific retrieval projection and constructing a dedicated index for each target ontology.
  • Conclusions: OntologyAligner provides a modular, extensible, and locally traceable solution for mapping biomedical text to ontology concepts.The framework supports phenotype integration, literature curation, cohort construction, and downstream biomedical computational tasks.

METHODS · Unified HPO normalization benchmark

The paper introduces PhenoNormBench, a standardized HPO normalization benchmark, and OntologyAligner, a three-stage framework for ontology-aligned retrieval, language-model reranking, and selective hierarchy-guided refinement.

  • Unified HPO normalization benchmark: Seven HPO normalization datasets were standardized and consolidated into PhenoNormBench for consistent evaluation across methods.The datasets vary in ontology versions and annotation standards, which complicates evaluation.
  • Unified HPO normalization benchmark: Each evaluation unit contains a phenotype phrase, Standard HPO ID, and Accepted HPO IDs, with predictions matching any accepted identifier counted as correct.Accepted HPO IDs include the Standard HPO ID and officially documented historical alt_ids.
  • Unified HPO normalization benchmark: 13,390 samples comprise PhenoNormBench, including 7,355 samples with multiple Accepted HPO IDs.Table 2 reports the sources and sizes of the seven constituent datasets.
  • Unified HPO normalization benchmark: OntologyAligner processes each phenotype phrase through ontology-aligned retrieval, language-model candidate reranking, and selective hierarchy-guided refinement before outputting the top-ranked HPO ID.OAR retrieves 20 candidates, LCR reranks them using terminological information, and HGR incorporates local hierarchy information when triggering conditions are met.
  • Unified HPO normalization benchmark: OAR adapts general-purpose embeddings to the target ontology by projecting and L2-normalizing TE3L embeddings of HPO labels and synonyms.TE3L produces 3,072-dimensional initial embeddings, followed by a shared bias-free linear projection.
  • Unified HPO normalization benchmark: OAR trains with same-concept variants as positives and selects 10 hard-negative neighboring concepts from the anchor’s 50 nearest variants.Concepts are scored by maximum variant-level cosine similarity, and InfoNCE increases positive similarity relative to hard negatives while identity regularization limits distortion.
  • Unified HPO normalization benchmark: After training, OAR indexes 44,814 variants from 19,836 HPO concepts in Chroma and returns the 20 highest-cosine-similarity concepts for LCR.Inference applies the same encoding, projection, and normalization procedure to the input phrase.

LLM Candidate Reranking

OntologyAligner’s LCR uses an LLM to compare and rerank OAR’s 20 candidates using terminology and definitions, selecting the most specific semantically equivalent concept. When top predictions differ, HGR uses local parent-child or sibling relations to refine ranking in ambiguous hierarchical cases.

  • LLM Candidate Reranking: LCR uses candidate terminology, preferred labels, synonyms, and definitions to perform fine-grained semantic comparison and fully rank OAR’s 20 candidates.It employs gpt-5.6-sol with temperature set to 0.
  • LLM Candidate Reranking: LCR selects the most specific HPO concept that is semantically equivalent to and fully captures the input phenotype phrase.The resulting ranking is passed to HGR.
  • Hierarchy-Guided Refinement: HGR activates when OAR and LCR Top-1 predictions differ and any compared candidates have a parent-child or sibling relationship.It compares the OAR Top-1 concept with candidates in the LCR Top-3.
  • Hierarchy-Guided Refinement: HGR provides related concepts, explicit is_a relations, and definitions in a local graph context to the LLM for reranking; otherwise, the LCR ranking is retained.Hierarchy is used because parent-child and sibling HPO concepts can be difficult to distinguish from labels, synonyms, and definitions alone.

context to refine the LCR ranking and generate the final prediction. · Experimental design and evaluation

The study evaluates phenotype normalization on PhenoNormBench against established baselines, with controlled implementation settings and supplementary analyses spanning datasets, difficulty strata, ablations, and model sensitivity. Ablation uses all 13,390 samples, while sensitivity analyses use a balanced 2,100-sample subset to vary candidate sizes, embeddings, and LLM backbones.

  • Experimental design and evaluation: OntologyAligner is compared with FastHPOCR, PhenoBERT, PhenoTagger, RAG-HPO-original, and RAG-HPO-TE3L on PhenoNormBench.Evaluation is limited to phenotype normalization; phenotype extraction is outside scope.
  • Experimental design and evaluation: FastHPOCR uses a rule-based index rebuilt with the current HPO release, while PhenoBERT and PhenoTagger retain their publicly released models and thresholds.These settings preserve each method’s stated implementation configuration.
  • Experimental design and evaluation: RAG-HPO-original uses bge-small-en-v1.5, whereas RAG-HPO-TE3L uses text-embedding-3-large for vector-index construction.Both variants upgrade the HPO assignment LLM from LLaMA-3.1 70B to gpt-5.6-sol and use the same Top-20 candidate set and prompt.
  • Experimental design and evaluation: All methods are additionally evaluated on seven constituent datasets, while OntologyAligner is stratified by phrase length, token-set Jaccard overlap, and target HPO concept depth.These analyses assess performance consistency across data sources and difficulty strata.
  • Experimental design and evaluation: 13,390 samples support stage-wise ablation across raw retrieval, OAR, OAR+LCR, raw retrieval+LCR+HGR, and full OAR+LCR+HGR configurations.The ablation quantifies the contribution of each framework stage.
  • Experimental design and evaluation: 2,100 balanced samples, formed by randomly selecting 300 per dataset with seed 42, support sensitivity analysis with equal dataset weighting.Candidate-size analysis varies k among 1, 3, 5, 10, and 20.
  • Experimental design and evaluation: Embedding sensitivity compares text-embedding-3-large, text-embedding-3-small, BioLORD, BioBERT, PubMedBERT, and ClinicalBERT with separately trained native-dimensionality OAR projections.Downstream LCR and HGR settings remain constant across embedding backbones.
  • Experimental design and evaluation: LLM sensitivity replaces the LCR and HGR backbone with Claude Opus 5, DeepSeek V4 Flash, or DeepSeek V4 Pro using TE3L-generated candidate sets.All LLMs are accessed through API requests, with model versions listed in Supplementary Table S1.

Generalization to other ontologies · Metrics and statistical analysis

OntologyAligner was evaluated beyond HPO on MONDO, MEDIC, and NCBITaxon under largely fixed experimental settings. Evaluation used Top-1 Accuracy with dataset-level aggregation, bootstrap confidence intervals, corrected paired tests, and predefined robustness analyses.

  • Generalization to other ontologies: OntologyAligner transferred to MONDO, MEDIC, and NCBITaxon using ontology-specific OAR projections and dedicated vector indexes.The experiments held Top-20 retrieval, gpt-5.6-sol, LCR reranking, and HGR procedures constant.
  • Generalization to other ontologies: The cross-ontology evaluations used 256 rare-disease phrases for MONDO, 960 disease phrases for MEDIC, and 767 organism phrases for NCBITaxon.These came from FGDD Disease, the NCBI Disease Corpus, and S800, respectively.
  • Metrics and statistical analysis: Top-1 Accuracy was the primary metric, with Macro Accuracy averaging the seven dataset accuracies and Micro Accuracy pooling all samples.Cross-ontology experiments reported Top-1 Accuracy only.
  • Metrics and statistical analysis: 20,000 percentile bootstrap replicates estimated 95% confidence intervals for accuracy.Micro Accuracy differences used the exact McNemar test with Holm correction, while Macro Accuracy differences used paired, dataset-stratified bootstrap with Holm correction.
  • Metrics and statistical analysis: Stage-wise ablation used the full evaluation set, whereas candidate-set, embedding-backbone, and LLM-backbone analyses used a balanced subset of 2,100 samples.Reference conditions were k=20, TE3L, and gpt-5.6-sol, with other settings unchanged; significance used the exact McNemar test with Holm correction.
  • Metrics and statistical analysis: 300 samples from each dataset made Macro Accuracy and Micro Accuracy identical on the balanced subset.Difficulty analyses stratified samples by phrase length, token-set Jaccard overlap with target-term variants, and target HPO concept depth.
  • Metrics and statistical analysis: PhenoNormBench and the study code are publicly available as resources for reproducing the benchmark and analyses.PhenoNormBench integrates seven standardized phenotypic datasets.

Supplementary Figure S1

Supplementary materials evaluate OntologyAligner across PhenoNormBench’s seven datasets and document the hierarchy-guided refinement prompts. The refinement uses HPO graph context alongside lexical and definitional evidence to distinguish highly similar candidates.

  • Dataset-level evaluation: Dataset-level Top-1 accuracy was compared between each baseline and OntologyAligner across all seven PhenoNormBench datasets using paired exact McNemar tests.Five within-dataset comparisons were adjusted with the Holm method; ** denotes P_adj<0.01 and *** denotes P_adj<0.001.
  • Hierarchy-guided refinement: Hierarchy-guided refinement supplies HPO graph context, including explicit is_a paths, together with preferred names, synonyms, and definitions.The prompt instructs the model not to infer clinical details absent from the original text or treat candidate order as evidence.
  • Hierarchy-guided refinement: The refinement prompt emphasizes graph relationships and definitions to distinguish highly similar HPO concepts while ranking all 20 candidates by semantic precision and completeness.Candidates are ordered from best to worst according to how precisely and completely they represent the original biomedical phrase.
Loading 2609.10055v1…