PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Enrichment Analysis with GraphRAG: Efficient Multi-Database ORA Extraction and Biological Interpretation

Guangcan Li, Shuyuan Zheng, Jiayao Liu, Shigeto Seno, Makoto Onizuka

arXiv:2610.30710v1cs.IRcs.DB

TL;DR

Conventional ORA leaves cross-database gene-set relationships fragmented and provides limited verification that generated interpretations are supported by enrichment evidence. GBOR-RAG connects terms through shared genes in a term–gene knowledge graph, retrieves compact evidence subgraphs for LLM interpretation, and reports faster ORA alongside improved evidence faithfulness and interpretation quality.

  • Problem

    Conventional ORA analyzes databases separately, and generated biological claims are rarely checked against the underlying enrichment evidence.

    Source: §I · p. 1

  • Method

    GBOR-RAG retrieves compact connected subgraphs from a multi-database term–gene knowledge graph to condition LLM interpretations, with an optional PubMed literature layer.

    Source: §I · p. 1, §IV · p. 3

  • Results

    Graph-based ORA achieves a median 37× speedup while closely reproducing conventional ORA, and GBOR-RAG records the highest claim-level faithfulness among compared systems.

    Source: §I · p. 2, A. Accuracy and Efficiency of… · p. 5

  • Takeaways & Limitations

    Retrieved gene-anchored evidence supports traceable interpretations, and adding PubMed literature further improves interpretation quality.

    Source: §I · p. 2, §VI · p. 7

    Evaluation relies on an LLM judge calibrated on small human-rated sets, and faithfulness measures consistency with evidence rather than biological correctness.

    Source: §VI · p. 7

Abstract

from arXiv · show

Over-representation analysis (ORA) is widely used to interpret gene-expression studies, but its outputs are often fragmented across gene-set databases and presented as flat term lists that obscure gene-level links among related enrichment signals. We propose Graph-Based Over-Representation Retrieval-Augmented Generation (GBOR-RAG), an integrated and traceable framework for ORA interpretation. GBOR-RAG represents heterogeneous gene-set databases as a unified term--gene knowledge graph, where shared genes connect related terms across sources. After graph-based ORA scoring, the framework retrieves a compact connected evidence subgraph that preserves both cross-database relationships and gene-level support. The subgraph conditions an LLM to generate biological interpretations, and we introduce a claim-level faithfulness metric to assess whether generated claims are supported by explicit evidence. Experiments show that graph-based ORA preserves conventional ORA results with high consistency (mean Pearson $r=0.997$) while reducing query time by a median of $37\times$ (up to $48\times$). The retrieved subgraphs average 36.8 nodes and achieve the highest overall integration score (4.12), and GBOR-RAG attains the highest claim-level faithfulness (0.90) among the compared systems. Adding a PubMed literature layer further improves the overall interpretation quality from 4.17 to 4.75.

I. INTRODUCTION

Conventional ORA leaves cross-database term relationships and gene-level support fragmented, while interpretation pipelines rarely check whether generated claims match enrichment evidence. GBOR-RAG connects terms through shared significant genes and retrieves evidence for LLM interpretation.

I. INTRODUCTION

  • Conventional ORA analyzes databases separately, leaving readers to compare terms, while existing integration tools may make gene-level links difficult to trace and raise scalability concerns.Ranked enrichment tables and term networks also leave biological interpretation to users, and generated claims are rarely checked against their evidence.Source: §I · p. 1
  • GBOR-RAG formulates integration as prize-collecting subgraph extraction, linking significant terms through shared significant genes rather than concatenating database-specific top terms.The retrieved graph can surface bridge signals not among the top-ranked results in any single database.Source: §I · p. 1
  • Mean Pearson r = 0.997 and median 37× speedup (up to 48×) show graph-based ORA reproduces conventional ORA while reducing query time across seven databases.The framework implements ORA on a term–gene knowledge graph.Source: §I · pp. 1–2
  • 36.8 nodes on average and an overall integration score of 4.12 accompany compact, fully connected retrieved evidence subgraphs.The subgraphs preserve connections among enriched terms through shared genes.Source: §I · pp. 1–2
  • GBOR-RAG achieves the highest claim-level faithfulness score among compared systems, at 0.90.The metric assesses whether generated interpretations are supported by retrieved evidence.Source: §I · p. 2
  • Adding a PubMed literature layer improves textual interpretation quality from 4.17 to 4.75.The layer adds gene–literature evidence to the knowledge graph.Source: §I · pp. 1–2

II. PRELIMINARIES

ORA tests whether a predefined gene set is enriched among genes of interest against a background universe. In expression studies, the paper applies the test separately to up- and down-regulated significant genes.

A. Over-Representation Analysis

  • ORA evaluates the one-tailed hypergeometric probability of observing at least k overlapping genes by chance; a small p-value marks a term as enriched.The test is described as the statistical core of widely used enrichment tools,.Source: A. Over-Representation Analysis · p. 2
  • The test uses background universe U, significant genes S, and term-annotated genes G, with N, n, K, and k denoting their sizes and overlap.Here, k is the number of genes in both G and S.Source: A. Over-Representation Analysis · p. 2
  • Significant genes are split by log-fold-change sign and ORA is run separately on S+ and S− against the same background U.The experiments use αDE = 0.05 and δ = 0, following.Source: A. Over-Representation Analysis · p. 2

B. Graph-Based Retrieval-Augmented Generation

GBOR-RAG addresses fragmented cross-database enrichment and weak claim verification by retrieving connected gene–term evidence for LLM interpretation. Its knowledge graph represents source-specific terms and genes, with shared genes linking evidence across databases.

III. RELATED WORK

  • Earlier cross-database tools provide term-centered networks or clusters, but fixed top-k selections can leave evidence fragmented or omit bridge terms.GBOR-RAG instead retrieves a connected, gene-anchored subgraph that explicitly represents cross-database links.Source: §III · p. 2
  • Claim support is rarely evaluated against the exact evidence provided to LLM-based interpretation systems, motivating GBOR-RAG’s evidence-linked faithfulness assessment.The method conditions the LLM on the retrieved gene–term subgraph and assesses claim-level faithfulness against that evidence.Source: §III · pp. 2–3

A. Multi-Database Knowledge Graph

  • The framework represents terms from multiple gene-set databases and genes as a bipartite graph joined by term–gene membership edges.Source-specific term nodes preserve database identity, while shared gene nodes connect evidence across sources.Source: A. Multi-Database Knowledge Graph · p. 3

B. Problem Formulation

The formulation targets integration of enrichment signals scattered across independent database rankings. It seeks evidence that is relevant, connected through shared genes, and compact without allowing database-specific p-value distributions to dominate.

B. Problem Formulation

  • Independent database rankings make it unclear whether terms from different sources support the same underlying mechanism.The stated goal is to integrate these separate enrichment signals across databases.Source: B. Problem Formulation · p. 3
  • A global ranking by raw p-value can favor particular resources because database size, term granularity, and annotation density bias significance distributions.Source: B. Problem Formulation · p. 3
  • The retrieved evidence subgraph is intended to prioritize relevant terms, connect databases through shared genes, and avoid unnecessary expansion.These objectives define relevance, connectivity, and compactness for cross-database integration.Source: B. Problem Formulation · p. 3

C. Graph-Based ORA

Graph-based ORA scores each term using its graph-neighbor gene set and ranks terms within each database. Retrieval then selects a compact connected evidence subgraph by balancing term relevance against connection costs.

C. Graph-Based ORA

  • Graph-based ORA reuses the hypergeometric test, replacing each predefined term gene set with that term node’s graph neighborhood.The universe is the intersection of genes in the knowledge graph and the measured genes.Source: C. Graph-Based ORA · p. 3
  • Significant genes are split by log-fold-change sign into positive and negative sets, and each term is scored against both sets with the same upper-tail test.Source: C. Graph-Based ORA · p. 3
  • Within-database ranking by nominal p-value supplies comparable candidate-term ranks; Benjamini–Hochberg adjustment within a database leaves the ranking unchanged.Terms are not pooled into one global ranking because their p-values are not comparable across databases.Source: C. Graph-Based ORA · pp. 3–4

D. Evidence Subgraph Retrieval

  • The prize-collecting objective favors high-prize terms while retaining only genes and edges needed to connect them, removing weakly relevant or isolated nodes.The selected evidence subgraph is solved as an unrooted, single-tree PCST and induced from the selected nodes.Source: D. Evidence Subgraph Retrieval · p. 4
  • Retrieval selects top-ranked candidate terms per database, then uses significance-based term prizes, small connector prizes for significant genes, and uniform term–gene edge costs.Per-database selection balances database representation; the prize cap limits an individual term’s contribution and normalization makes term prizes comparable with the gene connector prize.Source: D. Evidence Subgraph Retrieval · p. 4
  • Retrieval runs separately for positive and negative poles using the corresponding rankings and significant genes as connectors, and passes both subgraphs to interpretation.Source: D. Evidence Subgraph Retrieval · p. 4

E. Graph Retrieval Augmented Interpretation

The interpretation stage turns each retrieved subgraph into an evidence block for the language model. It preserves term–gene support and adds graph-structure cues while presenting positive and negative enrichment separately.

E. Graph Retrieval Augmented Interpretation

  • The language model is conditioned on both retrieved subgraphs and constrained to ground every statement in that evidence.Source: E. Graph Retrieval Augmented… · p. 4
  • Each evidence block orders terms by prize within each database and lists the significant genes connecting to each term.Hub genes and cross-database bridging genes add structural cues that a flat listing would hide.Source: E. Graph Retrieval Augmented… · p. 4
  • Positive and negative evidence blocks are presented in parallel so the model reports and contrasts processes enriched among each gene set.Both blocks represent enrichment signals, not a presence-versus-absence comparison.Source: E. Graph Retrieval Augmented… · p. 4

F. Advanced GBOR-RAG

GBOR-RAG adds a PubMed layer to the curated term–gene graph, assigning article prizes from how specifically cited genes overlap significant genes so literature can enrich retrieved evidence.

F. Advanced GBOR-RAG

  • The PubMed layer represents articles as PMID nodes connected to cited human genes, covering 26,309 genes and 26,506 in the combined graph.PMID nodes cannot be scored by ORA because they do not define annotated gene sets.Source: F. Advanced GBOR-RAG · p. 4
  • NPMI prizes scale article relevance to term prizes by measuring the association between an article citing a gene and that gene being significant.The score is bounded from −1 to 1 and rewards focused articles while down-weighting promiscuous articles, such as large screens and reviews.Source: F. Advanced GBOR-RAG · p. 4
  • Prize-collecting retrieval adds scored PMID nodes connected to significant genes they co-cite, yielding curated terms and genes accompanied by primary articles as citable mechanistic support.For the Alzheimer’s disease example, the literature-augmented method collects richer biological evidence than naive GBOR-RAG.Source: F. Advanced GBOR-RAG · p. 4

V. EXPERIMENTS

The experiments test whether graph-based ORA preserves conventional scoring while scaling across databases; it closely matches conventional ORA and is substantially faster.

V. EXPERIMENTS

  • The experiment evaluates whether graph-based ORA preserves conventional results and scales to many databases.The common setup fixes retrieval parameters and uses the same LLM generator and independent judge across experiments; judge scores are comparative rubric-based evidence, not absolute biological-correctness measures.Source: A. Accuracy and Efficiency of… · p. 5, §V · p. 5

A. Accuracy and Efficiency of Graph-based ORA

  • The evaluation uses 42 GEO2KEGG queries for efficiency and eight reference queries for accuracy, comparing graph-based ORA with conventional ORA under shared genes and size filters.Accuracy uses correlations, top-10 Jaccard overlap, and F1 for nominally significant terms; query time excludes the one-time graph build.Source: A. Accuracy and Efficiency of… · p. 5
  • Mean Pearson correlation was 0.997, and query time fell from about 15.6 s to 0.4 s across seven databases, a median 37× speedup and up to 48×.The speedup grows as databases are pooled; lower top-term overlap in DisGeNET and Open Targets mainly reflects library-version and background-universe differences.Source: A. Accuracy and Efficiency of… · p. 5, Figure 2 · p. 5

B. Quality of Cross-Database Integration

GBOR-RAG retrieves compact, connected evidence across databases, balancing relevance and integration; its advantage over top-k pooling is clearest in integration, bridging, and overall usefulness.

B. Quality of Cross-Database Integration

C. Claim-Level Faithfulness of Textual Interpretation

The paper evaluates whether generated claims are supported by the evidence supplied to the generator. GBOR-RAG achieves the highest reported claim-level faithfulness among the compared systems.

C. Claim-Level Faithfulness of Textual Interpretation

  • Claim-level faithfulness is the fraction of atomic interpretation claims entailed by the evidence supplied to the generator, averaged over queries.The independent LLM judge labels each claim as supported or unsupported; the metric measures evidence consistency, not biological correctness.Source: C. Claim-Level Faithfulness of Textual… · p. 6
  • The comparison uses 42 queries and an independent LLM judge to assess interpretations produced from each system’s own evidence.The judge is from a different model family than the generators.Source: C. Claim-Level Faithfulness of Textual… · p. 6
  • GBOR-RAG attains the highest claim-level faithfulness at 0.90, using about 27× less evidence than GeneAgent and only slightly more than MAPA.GeneAgent scores 0.71 and MAPA 0.29 on the subset where its module-clustering step applies; GBOR-RAG’s advantage is significant on shared queries.Source: C. Claim-Level Faithfulness of Textual… · p. 6, Table V · p. 6

D. Advanced GBOR-RAG: Literature-Augmented Retrieval

The PubMed layer improves interpretation quality, especially evidence anchoring and mechanistic insight. The conclusion cautions that faithfulness measures consistency with evidence, not biological correctness.

D. Advanced GBOR-RAG: Literature-Augmented Retrieval

  • Across 42 queries, a blind independent LLM judge rated interpretations on six dimensions; naive and advanced systems shared a generator and prose prompt, differing only in retrieved evidence.The dimensions were relevance and specificity, mechanistic insight, integration and synthesis, evidence anchoring, clarity, and overall quality.Source: D. Advanced GBOR-RAG:… · p. 6
  • On a blind calibration set of six queries and 12 interpretations, annotator agreement was pooled Spearman ρ = 0.67 and agreement with the LLM judge was pooled ρ = 0.69.For overall quality, human-average and LLM-judge agreement was pooled ρ = 0.74.Source: D. Advanced GBOR-RAG:… · p. 6
  • 4.17 to 4.75 (p < 10^-6): advanced GBOR-RAG raises overall interpretation quality, with largest gains in evidence anchoring (+0.75) and mechanistic insight (+0.39).Table VI reports mean LLM-judge ratings across 42 queries.Source: Table VI · p. 7, D. Advanced GBOR-RAG:… · p. 7
  • For GSE1297, PMIDs in prize-collecting retrieval let advanced GBOR-RAG collect richer biological evidence than naive GBOR-RAG.The example is shown for the negative subgraph in Fig. 3.Source: D. Advanced GBOR-RAG:… · p. 7

VI. CONCLUSION

  • The conclusion reports median 37× speedup over conventional ORA, compact connected subgraphs, and interpretations more faithful to evidence than existing systems.It also states that a PubMed literature layer further improves interpretation quality.Source: §VI · p. 7
  • Evaluation relies on an LLM judge calibrated on small human-rated sets, and faithfulness measures evidence consistency rather than biological correctness; a larger expert study remains future work.Source: §VI · p. 7

Related papers

Loading 2610.30710v1…