PaperlayerView exact revision on arXiv

Source-linked AI summary

Homo-RAG: Homology-Guided Retrieval-Augmented Generation for Cross-Species Gene Function Prediction

Azrin Sultana

arXiv:2608.25466v1cs.NEcs.AI

TL;DR

Functional annotation remains limited for many genes in non-model organisms. Homo-RAG combines homology-aware multi-hop retrieval with evidence-aware ranking, achieving an MRR of 0.980 in retrieval evaluation.

  • Problem

    Functional characteristics remain unexplored for an estimated 20–70% of genes, limiting annotation knowledge for non-model organisms.

  • Method

    Homo-RAG uses ortholog-guided multi-hop retrieval across structured biological databases and literature, followed by evidence-aware ranking for gene function prediction.

  • Results

    MRR reached 0.980 across candidate depths, placing the first relevant document at an average rank of 1.02.

  • Takeaways & Limitations

    The study establishes the empirical feasibility of combining homology-aware retrieval with evidence-aware ranking for gene-function prediction.

  • Takeaways & Limitations

    The final Evidence Confidence Score contains only one validated evidence-specific feature and uses fixed ranking values and open-source LLMs.

Abstract

from arXiv · show

The functional annotation of genes in non-model organisms remains a significant challenge in computational biology, with 20-70% of sequenced genes lacking characterized functions. Traditional homology-based methods are often costly and strongly dependent on high sequence similarity. This study presents Homo-RAG, a framework for large language model-based gene function prediction that integrates homology-guided multi-hop retrieval with evidence-aware ranking. The framework exploits biological relationships between zebrafish and human orthologs to guide evidence acquisition from ZFIN, UniProt, and PubMed through hybrid dense and lexical retrieval. An Evidence Confidence Score (ECS) integrates semantic relevance, entity matching, orthology information, source reliability, and literature association signals to refine the ranking of retrieved evidence. Extensive evaluation across 150 queries and 7,200 retrieved documents shows that evidence weighting parameter of lambda=0.50 improves NDCG@10 to 0.9879 and MRR to 0.99, while retrieving relevant evidence for 99.33% of queries. Furthermore, 80% of the retrieved documents are query-exclusive, indicating that evidence quality complements rather than replaces retrieval relevance. These findings establish Homo-RAG as a practical and robust framework for reliable, evidence-grounded gene function prediction in understudied organisms. The study addresses important limitations of conventional annotation pipelines while identifying opportunities for future improvements in evidence features and attribution mechanisms.

1 Introduction

Functional annotation is difficult because genomic data are expanding faster than gene functions are characterized, while biological knowledge is complex, current, and costly to incorporate into LLMs. Homo-RAG addresses these challenges through orthology-guided multi-hop retrieval, query reformulation, and evidence-confidence ranking.

  • Motivation: Millions of DNA and protein sequences have been generated, but many genes still lack explored functional characteristics.Understanding protein function supports research on ecological adaptations, evolutionary biology, and species specialization.
  • Prior approaches: Sequence-based methods transfer functional annotations from well-annotated homologous genes, but depend on comparisons with reference databases.BLAST is described as a method for identifying local similarity between query and subject DNA or protein sequences.
  • Motivation: RAG helps LLMs incorporate up-to-date genetic knowledge while reducing hallucinations and avoiding the high computational and financial costs of fine-tuning.The need for current information arises from newly discovered species, evolving genes, and changing genetic knowledge.
  • Homo-RAG framework: Homo-RAG uses ortholog relationships to guide evidence retrieval, first identifying the query gene and corresponding ortholog before retrieving curated UniProt knowledge.Retrieved UniProt evidence includes protein descriptions, functional annotations, pathway information, and related literature identifiers.
  • Homo-RAG framework: The framework combines multi-hop retrieval, query reformulation, and Evidence Confidence Scoring to address heterogeneous biological evidence and prioritize trustworthy supporting documents.Its retrieval links genes, orthologs, functional annotations, and biomedical literature, while reformulation reduces semantic and lexical mismatch.

2 Literature review

The literature review spans deep learning, multimodal information integration, and language-model approaches to protein and gene function prediction. It also highlights persistent challenges in contextual extraction, hallucination control, and the cost of fine-tuning.

  • Sequence-based prediction: Deep learning methods predict protein function from sequences using CNNs, RNNs, deep neural networks, and attention-based transformers.CNNs identify motifs, local patterns, and spatial relationships, while RNNs capture sequential information.
  • Information integration: DeepGraphGO and Graph2GO integrate broader biological information to address limitations of protein interaction-based and heterogeneous-data methods.DeepGraphGO incorporates data from all species into a single model, whereas Graph2GO uses multimodal graph-based representation learning.
  • Information integration: GoBERT explores the Gene Ontology graph with BERT to model relationships among gene functions.The approach annotates the Gene Ontology graph and applies BERT to decipher underlying functional relationships.
  • Sequence-based prediction: SPROF-GO uses a pretrained language model and self-attention pooling for efficient, alignment-free protein function prediction.Self-attention pooling focuses on important residues while extracting informative sequence embeddings.
  • LLM and RAG approaches: LLM-based approaches face challenges in extracting residue-specific context without hallucination, while fine-tuning remains costly and resource-intensive.RAG architectures have therefore been used to improve predictive precision; GeneRAG combines RAG with Maximal Marginal Relevance and reports a 39% improvement in answering gene questions over GPT-3.5 and GPT-4.

3 Methodology … 3.5 Hybrid Retrieval

Homo-RAG builds an evidence-grounded gene-function prediction pipeline from multisource biological data, structured retrieval documents, biomedical embeddings, and hybrid evidence retrieval. Its retrieval design combines semantic similarity, lexical matching, and exact-gene preference to capture both conceptual and entity-specific evidence.

  • 3 Methodology: Homo-RAG integrates homology-aware multi-hop retrieval, hybrid information retrieval, and retrieval-augmented generation for evidence-grounded gene function prediction in non-model organisms.For a target zebrafish gene, it identifies relevant gene profiles and corresponding human orthologs from structured biological knowledge.
  • 3.1 Dataset description: ZFIN, UniProt, and PubMed provide complementary structured gene information, curated functional knowledge, and literature-based biological evidence.ZFIN establishes zebrafish–human orthology relationships, UniProt enriches human ortholog context, and PubMed contributes textual biomedical evidence.
  • 3.2 Data Preprocessing: Preprocessing converts zebrafish information, orthology mappings, functional annotations, Gene Ontology data, UniProt annotations, and PubMed abstracts into consistent retrieval-ready representations.The process normalizes identifiers and gene symbols while handling whitespace inconsistencies, missing values, and duplicated records.
  • 3.3 Retrieval-Oriented Document Construction: Homo-RAG constructs separate gene-profile, UniProt knowledge, and PubMed document collections to represent complementary biological evidence.Each document retains a stable identifier and metadata such as gene identifiers, source type, and literature identifiers; the corpus contains approximately 17,224 gene-profile, 17,259 UniProt, and 32,909 PubMed documents.
  • 3.4 Biomedical Document Embedding: The system encodes biomedical documents with pritamdeka/S-PubMedBert-MS-MARCO, selected for biomedical-domain representations and retrieval-oriented training.The encoder supports semantic matching across gene, protein, and PubMed-derived evidence.
  • 3.5 Hybrid Retrieval: Dense retrieval identifies semantic relationships, while BM25 captures exact or near-exact relationships important for gene symbols, protein names, terminology, and identifiers.The normalized dense similarity and normalized BM25 scores are combined, with α ∈[0, 1] controlling the semantic retrieval contribution.
  • 3.5 Hybrid Retrieval: An exact-gene matching mechanism gives additional retrieval preference to documents explicitly referring to the target entity, improving precision for entity-specific evidence.The hybrid strategy captures both conceptual similarity and exact biomedical terminology rather than relying exclusively on dense or lexical retrieval.

3.6 Homology-Aware Multi-Hop Retrieval with query reformation · 3.7 Biological Evidence Ranking · 3.8 Evidence-Grounded Context Construction

Homo-RAG combines homology-aware multi-hop retrieval, evidence-quality ranking, and top-ranked context construction for evidence-grounded gene function prediction. The pipeline moves from a zebrafish gene to its human ortholog, integrates functional and literature evidence, scores documents with ECS, and supplies selected evidence to the language model.

  • 3.6 Homology-Aware Multi-Hop Retrieval with query reformation: Homo-RAG uses homology-aware multi-hop retrieval rather than mapping the original zebrafish query directly to documents.The retrieval trajectory uses biological entities discovered during preceding stages.
  • 3.6 Homology-Aware Multi-Hop Retrieval with query reformation: The retrieval process begins with a target zebrafish gene, identifies its human ortholog, and then retrieves ortholog-linked functional knowledge and literature evidence.The human ortholog provides a bridge to richer annotations and literature resources for better-characterized genes.
  • 3.6 Homology-Aware Multi-Hop Retrieval with query reformation: The implemented reformulation transitions from zebrafish gene context to human ortholog context, providing controlled biological query expansion rather than unrestricted rewriting.The expanded context supports retrieval of human functional knowledge and literature evidence.
  • 3.7 Biological Evidence Ranking: The Evidence Confidence Score distinguishes retrieval relevance from evidence quality by incorporating biological provenance and reliability.Documents with comparable retrieval relevance may differ substantially in evidential reliability.
  • 3.7 Biological Evidence Ranking: The final evidence score is normalized to [0, 1] and used in subsequent ranking and reranking stages.Evidence features are also normalized to [0, 1].
  • 3.7 Biological Evidence Ranking: Five complementary features contribute to the raw evidence score: semantic similarity, exact query-gene matching, ortholog matching, source reliability, and PMID linkage.The source reliability prior assigns 1.00 to gene profiles, 0.95 to UniProt, and 0.90 to PubMed.
  • 3.8 Evidence-Grounded Context Construction: Following evidence-aware reranking, Homo-RAG selects the highest-ranked evidence documents to construct the language-model context.The primary evaluation configuration uses K = 10.
  • 3.8 Evidence-Grounded Context Construction: The context pipeline is q →Dq →Rank(Dq) →Cq →LLM, retaining source metadata for evidence attribution and faithfulness analysis.The language model receives the highest-ranked evidence candidates rather than the entire retrieved corpus.

3.9 LLM-Based Gene Function Prediction · 3.10 Experimental Setup · 3.11 Performance Metrics

Homo-RAG evaluates five lightweight instruction-following LLMs under a standardized evidence-grounded prediction protocol and assesses the system across retrieval, evidence ranking, generation, attribution, and end-to-end dimensions. The experimental setup freezes gold evidence for evaluation while separating successful-response quality from generation reliability.

  • 3.9 LLM-Based Gene Function Prediction: Five lightweight LLMs receive the same biological query and evidence-grounded context to generate gene-function predictions.The evaluated models are Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct, TinyLlama-1.1B-Chat, Phi-3.5-mini-Instruct, and Gemma-2-2B-IT.
  • 3.9 LLM-Based Gene Function Prediction: The five models are evaluated with the same protocol to compare generation quality, evidence grounding, and consistency across compact architectures.The evaluation examines the suitability of lightweight generative models for evidence-grounded biological function prediction.
  • 3.10 Experimental Setup: 150 evaluation queries and 7,200 retrieved evidence rows were used in a GPU-enabled environment with an NVIDIA Tesla T4 GPU.The retrieved candidates included frozen gold evidence and non-gold candidates for ranking evaluation.
  • 3.10 Experimental Setup: Gold evidence membership was frozen before ranking experiments and withheld from the ranking mechanism to prevent evaluation information leakage.Generation used T = 0.2, nucleus sampling p = 0.9, and maximum length Nmax = 256.
  • 3.11 Performance Metrics: The evaluation covers retrieval effectiveness, evidence discrimination, ranking sensitivity, generation quality, faithfulness and attribution, and end-to-end performance.This decomposition separates the contributions of retrieval, evidence ranking, and generation.
  • 3.11.1 Retrieval Effectiveness: NDCG@K is the primary retrieval-ranking metric because ECS modifies the ordering of retrieved evidence, while ROC-AUC and PR-AUC assess gold-versus-non-gold discrimination.Precision@K, Recall@K, and Hit@K provide additional retrieval-effectiveness measures.
  • 3.11.1 Retrieval Effectiveness: Evidence ranking provides modest but measurable discrimination and is characterized as a ranking enhancement rather than a highly predictive standalone classifier.Sensitivity analysis evaluates how retrieval changes as evidence-confidence weight λ increases, including degradation when evidence confidence becomes excessively dominant.
  • 3.11.2 LLM Generation Quality: Generation quality is assessed only on successfully generated responses, while failures are evaluated separately rather than assigned artificial text-quality scores.Complementary metrics cover semantic correspondence, lexical diversity, generation efficiency, and operational reliability, including semantic similarity and BERTScore when available.

4 Result

Homo-RAG was evaluated as an integrated retrieval, homology-guided evidence-acquisition, reranking, and lightweight generation framework for gene-function prediction. Across 150 queries, retrieval produced a broad candidate pool and placed relevant evidence near the top of ranked lists.

  • 4 Result: The evaluation covered 150 queries, 7,200 retrieved evidence instances, and 951 gold evidence pairs, of which 672 appeared in the candidate pool.These retrieved gold pairs provided the basis for evaluating evidence retrieval.
  • 4 Result: All 150 queries retrieved the configured top-K documents at every hop, reaching 100% saturation with 450, 2,250, and 4,500 rows for K=3, 15, and 30, respectively.The exact row counts matched the theoretical maxima, with no empty or undersized candidate sets.
  • 4 Result: MRR of 0.980 indicated that the first relevant document appeared at rank 1.02 on average across candidate depths K = 1, 3, 5, 10, and 20.The ranking results showed relevant evidence consistently near the top and a saturation point relevant to cutoff selection.

K Precision Recall Hit NDCG

Homo-RAG achieves near-perfect top-k retrieval performance, with rapid saturation in hit rates and stable results across query types. Its full multi-hop and evidence-aware design substantially improves precision, recall, and NDCG over ablated or baseline configurations.

  • Hit-rate performance: 96.7% Hit@1 and 99.3% Hit@3 show that relevant evidence appears within the top three ranks for 149 of 150 benchmark queries.The 99.3% hit rate persists across deeper cutoffs, demonstrating strong discrimination by evidence-aware scoring.
  • Query-type stability: MRR ranges from 0.970 to 0.990 across query types, while Hit@10 is 1.000 for two categories and 0.980 for the remaining category.The three categories are Biological Function, Molecular Function, and Biological Processes.
  • Multi-hop ablation: Removing PubMed reduces Precision@10 from 0.419 to 0.170 and F1@10 from 0.533 to 0.286, while complete-system Recall@10 is 0.936 and Hit@10 is 0.993.PubMed supplies 62.5% of retrieved rows and is described as essential for candidate diversity and high recall.
  • Retrieval baselines: BM25-only yields P@10 = 0.35 and R@10 = 0.42, whereas FAISS-only yields R@10 = 0.62 and precision 0.28, exposing complementary retrieval trade-offs.Lexical retrieval favors exact gene-symbol overlap, while dense retrieval captures conceptual similarity but introduces spurious matches.
  • Evidence-aware reranking: NDCG@10 reaches 0.9879 and MRR 0.990 with the full ECS at λ = 0.30, exceeding Cross-Hop Support alone at 0.9842 and Source Reliability alone at 0.9831.The full three-hop hybrid backbone reaches NDCG@10 = 0.9805 and MRR = 0.980 before ECS component comparison.

5 Discussion

Homo-RAG combines hybrid biomedical retrieval with evidence-aware ranking to support gene function prediction, while its discussion highlights the complementary roles of retrieval stages and a quality-efficiency trade-off among language models. The study also identifies limitations involving evidence features, ranking models, and attribution mechanisms.

  • Retrieval and ranking: The BM25–FAISS hybrid retrieval architecture achieves NDCG@10 = 0.9805, MRR = 0.9800, and Hit@10 = 0.9933.These results support the complementary nature of lexical and semantic retrieval in biomedical applications.
  • Retrieval and ranking: Removing PubMed causes Precision@10 to collapse to 0.170, showing that PubMed supplies primary candidate diversity for high recall and near-perfect hit rates.Gene Profile provides focused, high-precision evidence, while UniProt expands functional coverage.
  • Retrieval and ranking: At λ = 0.50, evidence-aware reranking achieves precision@10 = 0.8856 versus the baseline 0.4186 without degrading semantic matching.Source reliability from curated gene profiles and peer-reviewed PubMed provides a complementary signal to semantic relevance.
  • Generation quality and efficiency: Phi-3.5-mini-Instruct reaches BERTScore = 0.87, Semantic Similarity = 0.84, Distinct-1 = 0.19, and Distinct-2 = 0.44, whereas TinyLlama-1.1B-Chat records 0.75 and 0.72 for the first two measures.The comparison indicates a quality-efficiency trade-off and that model size alone does not predict practical usability.
  • Limitations and future work: The implementation contains only one validated evidence-specific feature in the final ECS and uses fixed ranking values and open-source LLMs rather than frontier models.Proposed future directions include richer biological evidence features, claim-level attribution, cross-source consistency verification, and query-level aligned artifacts.

6 Conclusion

Homo-RAG combines homology-aware multi-hop evidence expansion, evidence-aware reranking, and iterative query reformulation for gene function prediction in non-model organisms. Evidence weighting improves ranking when used complementarily, while excessive weighting degrades performance and motivates learned evidence integration.

  • Core contributions: Recall@10: 0.73 →0.94 and NDCG@10: 0.83 →0.98 followed the homology-aware three-hop knowledge graph’s expansion from gene profiles through UniProt to PubMed.This progressive evidence expansion delivered the framework’s single largest performance gain.
  • Core contributions: ECS reranks evidence by integrating source reliability and cross-hop evidence consistency.The framework also reformulates queries after each retrieval to extract accurate information from external documents.
  • Ranking findings: λ = 1.0 substantially degraded performance, showing that evidence-aware ranking should complement rather than replace the original retrieval relevance signal.The original retrieval signal should remain the dominant ranking factor.
  • Limitations and future work: Homo-RAG currently uses only evidence features, without graph-based features, which may limit reliable score generation.This limitation motivates extending ECS with a learning-to-rank model incorporating multiple evidence dimensions and data-driven feature weighting.
  • Significance: Homo-RAG establishes a unified framework for homology-aware biomedical retrieval that connects semantic relevance with biological evidence quality.The framework is presented as accessible and applicable to gene function prediction in non-model organisms.

Funding

The research received no specific grant from public, commercial, or not-for-profit funding agencies.

  • No specific grant supported the research from public, commercial, or not-for-profit sectors.

Ethics approval and consent to participate · Materials availability

The study relied on publicly available biological databases and scientific literature datasets, without involving human participants, animals, or personal-data collection; therefore, ethics approval and consent were not required.

  • Materials availability: Publicly available biological databases were used in the study.
  • Materials availability: The study used publicly available scientific literature datasets.
  • Ethics approval and consent to participate: The study did not involve human participants.
  • Ethics approval and consent to participate: The study did not involve animals.
  • Ethics approval and consent to participate: The study did not collect personal data.
  • Ethics approval and consent to participate: Ethics approval and consent to participate were not required.
Loading 2608.25466v1…