Source-linked AI summary
Expert Knowledge & Machine Understanding: Bridging Reactome's Ontology with LLM Semantic Embeddings
Susanna Bravi, Riccardo De Luca, Rosa Sicilia, Christine Nardini, Mario Santoro
TL;DR
Manual curation makes Reactome reliable but difficult to scale, and it remains unclear whether textual descriptions encode higher-order biological relationships. The paper reconstructs a Homo sapiens semantic hierarchy from Reactome descriptions using SPECTER2, modified clustering, and graph reconstruction, then compares it with the curated hierarchy. The reconstructed structure reasonably reproduces Reactome’s organization, with strong global topological alignment and statistically significant bootstrap results.
Problem
The study addresses whether human-written Reactome descriptions can reconstruct expert-defined biological hierarchy despite manual curation’s scalability challenge.
Method
The workflow extracts the Homo sapiens Reactome Hierarchy, embeds titles and descriptions with SPECTER2, constructs a Semantic Hierarchy using clustering and graph reconstruction, and compares the graphs.
Results
The reconstructed Semantic Hierarchy reasonably reproduces the reference Reactome structure, with Laplacian Spectral Distance 0.085 and bootstrap significance p < 0.005.
Takeaways & Limitations
Human-written Reactome metadata contains enough structural signal to support preliminary reconstruction of the curated global pathway hierarchy.
Takeaways & Limitations
The evaluation covers one species, uses a limited set of comparison metrics, and remains a preliminary computational study whose generalisability is unassessed.
Abstract
from arXiv · showhide
Biological knowledgebases like Reactome provide high-quality pathways that include biological elements' relationships and textual descriptions (metadata). The quality of such pathways is granted by manual curation, that presents, however, significant scalability challenges. Lately, numerous NLP tools have been proposed to cope with this issue, leveraging textual information to automatically expand biological knowledgebases. However, little exploration has been done so far to assess whether relationships among textual descriptions mirror higher order biological relationships. This study explores whether human-written descriptions in Reactome can be used to infer the experts' defined global hierarchical structure. To test this, we extracted from Reactome the Homo Sapiens hierarchy of pathways and their reactions (Reactome Hierarchy), and used textual metadata to reconstruct a Semantic Hierarchy, combining a sentence transformer model (SPECTER2) with a modified agglomerative nesting algorithm and a graph reconstruction algorithm. Quantitative (Laplacian Spectral Distance and Bootstrapping) and qualitative (global topological metrics) analyses confirm our hypothesis and indicate that the global hierarchical structure of pathways can be inferred by experts textual metadata.
1 Introduction
Reactome combines manually curated biological hierarchies with human-written textual metadata, but manual curation creates scalability challenges. This study tests whether those descriptions can reconstruct expert-defined biological structure through semantic embeddings and graph comparison.
- Reactome and the research gap: Reactome organizes interconnected Reactions into hierarchically structured Pathways while attaching titles and human-written summaries to both.The Reactome Hierarchy is represented as a large interconnected graph of Pathways and Reactions.
- Reactome and the research gap: Manual curation provides high-fidelity pathway data but presents a significant scalability challenge for expanding biological knowledgebases.NLP and LLM methods motivate more automated use of textual metadata for knowledge integration and interaction.
- Prior work and contribution: Prior work used language models for kinetic modeling, computer-assisted annotation, and conversational retrieval rather than reconstructing existing biological hierarchy.These efforts addressed related knowledgebase applications but not structural reconstruction from existing textual metadata alone.
- Prior work and contribution: The study evaluates whether semantic embeddings from existing Reactome titles and descriptions can recover hierarchical relationships defined by experts.It compares a reconstructed Semantic Hierarchy with the expert-curated Reactome Hierarchy using graph-theoretic evaluation.
2 Data and Methods
The workflow extracts the Homo sapiens Reactome Hierarchy, embeds textual descriptions with SPECTER2, constructs a Semantic Hierarchy through modified clustering and graph reconstruction, and compares the two graphs.
- Data extraction and representation: The study extracts Homo sapiens Pathway and Reaction nodes, hierarchical relationships, and textual descriptions from Reactome.All Event-node subclasses are included and aggregated into a graph with descriptions as node attributes.
- Data extraction and representation: A tabular dataset records each node’s Reactome ID, entity type, title, and textual description.The dataset supplies the Name and Description fields used for sentence embeddings.
- Semantic embedding and preprocessing: SPECTER2 generates biomedical semantic embeddings from the textual descriptions of Pathways and Reactions.The model is a sentence transformer pretrained on scientific titles and abstracts using contrastive learning over a citation graph.
- Semantic embedding and preprocessing: The Semantic Hierarchy uses cleaned, concatenated node text, with long descriptions split into overlapping sentences to preserve context within the 512-token limit.Bibliographic citations are removed, and a [SEP] marker separates node names from descriptions.
- Modified hierarchical clustering: The modified AGNES algorithm permits simultaneous multi-node merges, producing a non-binary hierarchy rather than only pairwise binary merges.Merge heights and a data-driven threshold determine whether existing nodes are absorbed or retained as intermediate branches.
- Graph reconstruction: Graph reconstruction replaces mathematical Clusters with appropriate Pathway nodes using post-order depth-first traversal and Reactome-derived ancestor relationships.This resolves nested subclusters bottom-up and connects Reactions and Sub-Pathways to valid parents.
- Hierarchy comparison: Similarity between Semantic and Reactome Hierarchies is assessed with adjacency-matrix Edit Distance, Laplacian Spectral Distance, topology measures, and degree-preserving bootstrap rewiring.The bootstrap generates 500 null models to test whether observed global alignment exceeds randomized structure.
3 Results
The semantic hierarchy closely preserves Reactome’s reference structure across graph size, local connectivity, and global topology, with bootstrap results supporting non-random alignment.
- Reference hierarchy: 18,732 nodes and 19,613 edges comprise the Homo sapiens Reactome Hierarchy, including 29 TopLevelPathway roots.The hierarchy contains 2,848 Pathways and 15,884 Reactions.
- Semantic reconstruction: The Semantic Hierarchy preserves the node count with 18,703 edges, while embedding generation, clustering, and reconstruction complete in approximately 2 minutes, 46 seconds, and 6 seconds.
- Local structure: 216.88 Frobenius distance represents approximately 1.16% of the theoretical maximum for graphs of this order.This measures local edge-level divergence between the Semantic Hierarchy and Reactome Hierarchy.
- Global topology: Density remains 0.00011 in both hierarchies, while average degree is 2.00 in the Semantic Hierarchy versus 2.09 in Reactome.
- Global topology: Laplacian Spectral Distance is 0.085 versus the normalized upper bound ln(2) ≈ 0.693, and bootstrapping reports p < 0.005.The spectral density of the semantic hierarchy nearly overlaps Reactome’s, preserving the dominant peak at λ = 1.
4 Conclusion
The study constructs a Semantic Hierarchy from SPECTER2 embeddings of Homo sapiens Reactome metadata and finds it can reasonably reproduce the manually curated Reactome structure. The work is preliminary, with computational and topological evaluation limited to one species and requiring further biological validation and broader testing.
- 4 Conclusion: The reconstructed Semantic Hierarchy reasonably reproduces the manually curated Homo sapiens Reactome structure using textual embeddings, clustering, and graph reconstruction.The workflow compares a Reactome Hierarchy with a Semantic Hierarchy derived from names and descriptions.
- 4 Conclusion: The study is a feasibility and accuracy assessment rather than an exhaustive evaluation of the workflow.The authors characterize the contribution as a preliminary step within a broader research agenda.
- 4 Conclusion: Evaluation is currently computational and topological, with biological validation by domain experts planned for future work.The planned validation would assess the results from a biological perspective.
- 4 Conclusion: Generalisability remains unassessed because the results come exclusively from Reactome and the Homo sapiens hierarchy.Future work also proposes additional graph-similarity metrics and node-to-node or graph-alignment analyses.
- 4 Conclusion: The Semantic Hierarchy could support testing and refining automatic curation tools while preserving quality and reducing manual-curation burden.The authors connect this potential use to identifying less-curated areas and providing feedback to curators.