Source-linked AI summary
Filling holes in science draws collective attention, but most higher-order holes remain unexplored
Jiajie Luo, James A. Evans
TL;DR
Science leaves a growing set of higher-order combinations unexplored, raising questions about which holes researchers fill and whether higher-order gaps build on lower-order ones. The paper uses concept embeddings and persistent homology to detect holes and identify works that fill them. It finds that hole-filling works are often more novel and cited, while higher-order holes grow combinatorially and their filled fraction collapses, with citation patterns differing across fields.
Problem
As scientific knowledge expands, it remains unclear which gaps between concepts are filled, how attention relates to hole order and size, and whether unexplored higher-order combinations represent a loss.
Method
The paper represents knowledge with concept embeddings, applies persistent homology to detect holes of increasing order, and identifies works whose concepts fill those holes.
Results
Higher-order holes grow combinatorially while their filled fraction collapses; hole-filling works are more novel and cited on average, with larger higher-order gaps most rewarded in empirical fields but not formal or design fields.
Takeaways & Limitations
Bridging anticipated holes can indicate where science converges and attracts collective attention, while most higher-order combinations remain unexplored.
Takeaways & Limitations
The paper leaves open whether higher-order combinations are cumulative on lower-order ones, so the significance of their being unexplored remains uncertain.
Abstract
from arXiv · showhide
Much scientific discovery involves filling holes between ideas and arguments that unleash techno-scientific advance. Representing knowledge as high-dimensional concept embeddings, we use persistent homology to detect holes of increasing order, from gaps between disconnected ideas to higher-order cavities, and identify the research works that fill them. We find two empirical asymmetries. Researchers who fill anticipated holes are poised to draw collective attention by staging outsized novelty and foresight, indicating that bridging holes anticipates where science will converge, most strongly in empirical fields and least in formal and design fields. Yet as knowledge grows, higher-order holes explode while the fraction science fills collapses, leaving most higher-order combinations unexplored. These results call for a richer science of holes, and mark a frontier where contemporary AI might help fill the high-dimensional gaps human science opens.
Results
Higher-order holes become increasingly numerous as disciplines accumulate concepts, while their fill rates decline sharply. Works that bridge holes tend to be more novel and cited, especially in empirical fields, although formal and design fields show weaker or reversed citation advantages.
- Mechanisms of filling: Simulated papers usually fill only a tiny fraction of the holes filled by actual papers, except when simulations combine nearby concepts.Constraining simulated papers too strongly toward similar concepts eventually lowers their fill rate.
- Fill rates: As hole size increases, fill rates decline across dimensions; dimension-0 rates remain near 0.65–0.8, whereas dimensions 1 and 2 approach zero.Large dimension-1 holes are rarely filled, and dimension-2 holes are very rarely filled across disciplines.
- Novelty and attention: Empirical fields receive their strongest citation advantage for works closing large gaps, whereas formal fields concentrate citation advantages in smaller gaps.The field split widens with hole order, and citation rewards for larger gaps depend on whether a field is empirical or formal.
- Novelty and attention: Hole-filling works are more surprising than general scientific work across dimensions, and content surprise rises with hole order in natural sciences and several other fields.Filling larger gaps therefore generally corresponds to greater content novelty, even when it does not yield additional citations.
Discussion
Using persistent homology to identify holes in concept spaces, the paper finds that filling anticipated gaps attracts attention, but recognition and citation rewards vary across fields. Higher-order holes proliferate faster than researchers fill them, leaving most combinations unexplored and motivating a quantitative science of holes.
- Approach: Persistent homology identifies perceivable, anticipated holes in the concept space of scientific disciplines and links them to published works that fill them.The analysis embeds concepts across 19 disciplines and studies which holes were filled and what happened to the filling works.
- Attention and novelty: Works that fill holes are, on average, more cited and more surprising, but the citation advantage is widespread rather than universal.The advantage is most consistently observed for surprise across disciplines and hole dimensions.
- Field differences: In empirical sciences, citation premiums grow with hole order and size, whereas in formal and design fields they concentrate in small gaps and vanish or reverse for higher-order holes.Mathematics is the limiting case, with no citation premium for filling higher-order holes.
- Field differences: Large holes are recognized as novel almost everywhere, but whether that novelty earns attention depends on the field.Surprise rises with hole order on both sides of the empirical–formal divide, while citation patterns differ across fields.
- Unfilled frontier: Higher-order holes grow combinatorially as concepts accumulate, while the number filled grows much more slowly, leaving most higher-order combinations unattempted.Whether this constitutes a loss depends on whether higher-order combinations are cumulative from lower-order ones or instead emergent and non-additive.
- Implications: The results support a quantitative science of holes that models which gaps will be filled and distinguishes those reachable through incremental accumulation from those that are not.The authors suggest that machines could probe emergent combinations while human judgment selects and interprets the results.
Materials and Methods
The study builds discipline-year concept embeddings from OpenAlex works and applies persistent homology to identify and track holes across dimensions. It then links holes to works that combine their constituent concepts and estimates how often holes are filled.
- Data: The dataset contains over 250 million academic works, with concepts organized hierarchically from broad disciplines to increasingly specific topics.The analysis uses a December 2024 OpenAlex snapshot and examines 19 level-0 disciplines.
- Data: For each discipline-year pair, works published by that year are represented as bags of level 2–5 concepts for embedding.Works associated with multiple disciplines contribute to each relevant discipline’s set.
- Embedding construction: Word2Vec models produce 100-dimensional concept embeddings for each discipline from 2005 through 2019.Concepts must occur at least 10 times; this retains at least 97.7% of concepts in every discipline-year.
- Persistent homology: Persistent homology detects holes as cycles that are not boundaries and records their birth and death across scale-based simplicial-complex filtrations.The Betti number counts independent holes, while birth and death values indicate when holes form and fill.
- Persistent homology: The analysis computes persistent homology in dimensions 0 and 1 for every embedding and in dimension 2 for most disciplines.Dimension-2 computation is omitted for biology, chemistry, computer science, engineering, medicine, and physics because of computational limitations.
- Hole filling: A work fills a hole when the concepts forming its death simplex are a subset of the concepts associated with that work.This attribution links each filled hole to specific works rather than aggregate interaction statistics.
- Hole filling: The filled proportion is estimated within 5-percentile bins of hole death values and then averaged across years.The procedure is applied separately for each discipline and hole dimension.
Supplementary Text
Higher-order holes become exponentially more numerous while their fill rates decline, and the reception of hole-filling work differs sharply between empirical and formal or design fields. Across dimensions, hole-filling papers tend to show greater citation, surprise, and prescience, but these advantages vary by metric and discipline.
- Hole abundance and fill rates: The number of holes increases exponentially with dimension, while the fraction filled decreases universally in dimension 2.The dimension-2 fill-rate decrease is reported for all 19 disciplines.
- Hole geometry: Filled holes have smaller min–max ratios than holes generally, suggesting that successful filling combines concepts at comparable distances while tolerating some geometric unevenness.The pattern is described for higher-dimensional holes involving three concepts in dimension 1 and four in dimension 2.
- Surprise and citation: Content surprise generally rises with hole dimensionality, including in mathematics and philosophy, even where higher-dimensional filling earns no additional citation.Economics and history are identified as exceptions to the broader content-surprise pattern.