PaperlayerView exact revision on arXiv

Source-linked AI summary

Filling holes in science draws collective attention, but most higher-order holes remain unexplored

Jiajie Luo, James A. Evans

arXiv:2608.28822v1cs.CYcs.DLcs.SI

TL;DR

Which anticipated holes in scientific disciplines are filled by published work, and what happens to the works that fill them? Using persistent homology to identify these holes, the paper finds that higher-order holes grow combinatorially while the fraction filled collapses.

  • Problem

    The paper asks which anticipated holes in scientific disciplines are filled by published work and what becomes of the works that fill them.

    Source: Discussion · p. 12

  • Method

    The paper uses persistent homology to identify perceivable, anticipated holes in disciplinary concept spaces and examine published works that fill them.

    Source: Discussion · p. 12

  • Results

    Higher-order holes grow combinatorially while the fraction filled collapses.

    Source: Discussion · p. 13

  • Takeaways & Limitations

    The findings motivate a richer science of holes.

    Source: Discussion · p. 13

    Whether unfilled higher-order combinations represent a real loss depends on whether they are cumulative on lower-order combinations.

    Source: Discussion · p. 14

Abstract

from arXiv · show

Much scientific discovery involves filling holes between ideas and arguments that unleash techno-scientific advance. Representing knowledge as high-dimensional concept embeddings, we use persistent homology to detect holes of increasing order, from gaps between disconnected ideas to higher-order cavities, and identify the research works that fill them. We find two empirical asymmetries. Researchers who fill anticipated holes are poised to draw collective attention by staging outsized novelty and foresight, indicating that bridging holes anticipates where science will converge, most strongly in empirical fields and least in formal and design fields. Yet as knowledge grows, higher-order holes explode while the fraction science fills collapses, leaving most higher-order combinations unexplored. These results call for a richer science of holes, and mark a frontier where contemporary AI might help fill the high-dimensional gaps human science opens.

Results

The Results examine how hole counts and fill rates vary across disciplines and hole order, and how hole-filling works relate to citations and surprise. They report that higher-dimensional holes multiply as the number filled falls, while citation advantages and surprise patterns vary across fields and hole order.

Holes in embedding spaces

  • For nearly all disciplines, hole counts grow exponentially with dimension while filled-hole counts decline exponentially.The decline is especially pronounced for higher-order holes.Source: Holes in embedding spaces · p. 6
  • Hole counts scale with concept counts with power-law exponents 1 (k=0), 1.37 (k=1), and 1.87 (k=2); filled-hole exponents are 1.1, 1.51, and 1.78, respectively.The exponents for both counts rise with dimension.Source: Holes in embedding spaces · p. 8
  • Fill rates reflect how papers combine concepts rather than topology: scientists stay within disciplinary commitments yet fill modest nearby holes to produce novel work (3).Source: Holes in embedding spaces · p. 8
  • Nine null models usually produced only a tiny fraction of observed fill rates; models favoring nearby concepts matched empirical rates, but excessive similarity constraints reduced filling.Source: Holes in embedding spaces · p. 8
  • Larger holes are less likely to be filled across dimensions; dimension-0 rates remain 0.65–0.8, while dimension-1 and -2 rates approach zero at higher death percentiles.This pattern is consistent across disciplines.Source: Holes in embedding spaces · p. 9

Works that bridge holes

  • Hole-filling works average higher citation counts across dimensions; hit-paper likelihood rises for zero-order gaps across fields and first-order gaps in almost all, but rises with hole order only in empirical fields and falls in formal and design fields.The consistency of the citation premium varies by dimension.Source: Works that bridge holes · pp. 9–10
  • Top-quintile citation enrichment rises from roughly 1.5 to 2.6 across dimensions 0–2 in empirical and natural sciences, but falls from 1.2 to 0.9 in formal and design fields.The formal and design fields cross below the no-enrichment line by dimension 2.Source: Works that bridge holes · p. 11
  • For each 100 death percentiles, dimension-0 citation-enrichment slopes average +0.50 in empirical fields and −0.56 in formal fields, with the citation optimum at large gaps in empirical fields but small gaps in formal fields.Dimension 1 repeats the split more weakly, at +0.52 versus −0.25.Source: Works that bridge holes · pp. 11–12
  • Across dimensions 0–2, hole-filling works are more surprising than the full population of published work on average, with only a few discipline-specific exceptions.Exceptions are business and economics for dimension-2 content surprise, and business and environmental science for context surprise.Source: Works that bridge holes · p. 12
  • Content surprise rises with hole order on both sides of the empirical–formal divide, while context surprise falls from dimension 0 to 1 in natural and engineering sciences.Context-surprise death dependence, where present, is led by engineering and the physical and life sciences.Source: Works that bridge holes · p. 12

Discussion

Works that fill anticipated holes tend to attract attention, while higher-order holes proliferate as concepts accumulate and the fraction filled collapses. The attention advantage varies by field, and many higher-order combinations may remain beyond incremental science.

Discussion

  • Across nearly every discipline, works that close holes are more likely than peers to rank among the year's most surprising and are usually among the most discussed and referenced later.The surprise advantage holds across essentially all disciplines and all three hole orders; the citation advantage is widespread but not universal.Source: Discussion · p. 13
  • In empirical sciences, citation premiums grow with hole order and size; in formal and design fields, they fade or reverse for larger, higher-order holes.The formal/design pattern includes mathematics, economics, business, political science, philosophy, art, and, in dimension 1, computer science and engineering; mathematics has no higher-order citation premium.Source: Discussion · p. 13
  • Content surprise rises with hole order on both sides of the field divide, and filling large holes is recognized as novel almost everywhere, even when that novelty is not rewarded with attention.The divide is specific to attention: neither surprise nor prescience sorts disciplines in the same way as citation.Source: Discussion · p. 13
  • As fields grow, higher-order holes proliferate by power-law scaling with an exponent that increases with order, while the number filled grows far more slowly.The overwhelming majority of higher-order combinations are never attempted.Source: Discussion · p. 14
  • Whether unfilled higher-order holes represent a lasting loss depends on whether their value accumulates from lower-order combinations or emerges only when all elements are joined.Incremental science may reach cumulative combinations by filling lower-order gaps, but emergent combinations may lack any profitable partial step and remain outside that path.Source: Discussion · p. 14
  • Emergent higher-order combinations are hard for human science to attempt and evaluate because experiments are costly and reviewers fluent across disparate fields are rare.The breadth that makes such combinations novel can lead myopic reviewers to read them as arbitrary rather than profound.Source: Discussion · p. 15
  • A quantitative science of holes could model filling probabilities and accumulation dynamics, and assess how far realized discoveries lag the combinatorial opportunities concepts afford.The reported scaling laws between holes, concepts, and works that fill them are first elements of this research program.Source: Discussion · p. 15
  • AI could search high-dimensional gaps human incentives and cognition skip, while human judgment selects and interprets the results.The proposed division of labor targets emergent combinations that demand broad expertise, are costly to attempt, and lack an incremental gradient.Source: Discussion · pp. 15–16

Materials and Methods

The Materials and Methods describe building yearly, discipline-specific concept embeddings and using persistent homology to identify holes and the works that fill them, then assessing hole-filling papers through filling rates, citation and novelty metrics, and comparisons of surprise indices.

Data

  • OpenAlex’s December 2024 snapshot supplies over 250 million works, whose hierarchical concepts define 19 level-0 disciplines and their descendant topics (33).Concepts range from broad disciplines at level 0 to increasingly specific topics at levels 2–5.Source: Data · p. 17

Embeddings of Concept Spaces

  • For each discipline and year from 2005–2019, the study trains 100-dimensional Word2Vec embeddings on cumulative bags of level 2–5 concepts, excluding concepts occurring fewer than 10 times.The frequency floor retains at least 97.7% of concepts in every discipline–year.Source: Embeddings of Concept Spaces · p. 17

Persistent Homology

  • Persistent homology on each embedding’s Vietoris–Rips complex tracks holes in dimensions 0 and 1, and in dimension 2 for most disciplines, with six large disciplines excluded for computational reasons.A hole’s death simplex represents the concepts that fill it; a work is associated with the hole when it contains those concepts.Source: Persistent Homology · p. 19

Holes Filled

  • To estimate filling proportions, the study groups holes into 5-percentile bins by death value for each discipline and year, calculates each bin’s share with associated works, and averages across years.Source: Holes Filled · p. 20

Assessing Hole-Filling Papers

  • For each discipline and year, the authors compare the share of hole-filling papers at or above the pth citation percentile with the share of all papers meeting that threshold.The ratio indicates the relative over- or under-representation of hole-filling papers among highly cited works.Source: Assessing Hole-Filling Papers · p. 20
  • For content- and context-surprise, the authors use the percentile ranks already provided by (35), without additional percentile computation.They apply a similar analysis to these indices as for citation counts.Source: Assessing Hole-Filling Papers · p. 21

Supplementary Text

The Supplementary Text details analyses of hole counts, fillability, geometry, and methods for identifying filled holes, including randomized and semantically similar concept-set comparisons. It also examines how hole-filling relates to citation patterns, attention, surprise, and prescience across dimensions and disciplines.

Concept Retention Under the Frequency Threshold

  • Concept retention exceeds 97.7% in every discipline–year during 2005–2019 and 99.3% from 2010 onward, with no empirical/formal division difference.Formal and design fields retain as large a share of concepts as empirical fields, so differential pruning does not explain their contrasting citation premium at higher hole orders.Source: Concept Retention Under the Frequency… · p. 26

Holes Filled by Discipline

  • Hole counts increase exponentially with dimension except in environmental science, whereas filled-hole counts decrease exponentially in every discipline; largest fields’ dimension-2 counts are extrapolated.Environmental science has fewer than 500 concepts; dimension-2 persistent homology is not computed directly for biology, chemistry, computer science, engineering, medicine, or physics.Source: Holes Filled by Discipline · p. 26
  • Small gaps fill faster than large ones in 15 of 19 disciplines in dimension 0, in all 19 in dimension 1, and in every discipline analyzed in dimension 2.The four dimension-0 exceptions are environmental science, business, economics, and geology.Source: Holes Filled by Discipline · p. 27
  • Dimension 0 has more concave than convex fillability curves, while in dimension 2 every curve is very convex except environmental science, the sole concave case.Source: Holes Filled by Discipline · p. 27
  • In dimension 1, ratios for all and filled holes are predominantly 0.8–1; in dimension 2, all-hole ratios are predominantly 0.8–1, while filled-hole ratios are 0.6–0.8, and filled holes average smaller ratios overall.Source: Holes Filled by Discipline · pp. 27–28
  • At the 25% citation threshold, hit-paper likelihood rises with hole dimension only in empirical fields, while formal and design fields show the opposite pattern.All fields show increased hit papers for bridging zeroeth-order gaps and filling first-order holes.Source: Holes Filled by Discipline · p. 28
  • The same directional pattern holds at 10%, 5%, and 1%, but higher-than-baseline likelihood for first- and second-order holes declines as citation thresholds tighten.At the 1% threshold, filled H1 holes have higher-than-average citation likelihood in every field except engineering and art.Source: Holes Filled by Discipline · p. 28

Holes Filled by Simulated Papers

  • Synthetic-paper null models preserve the observed number of concepts per paper and fixed embedding topology, then vary other paper features to compare fill rates with real papers.The uniform-random model preserves paper sizes but neither concept popularity nor relationships between concepts; the described frequency-preserving resample also preserves concept frequencies.Source: Holes Filled by Simulated Papers · pp. 28–29
  • Except for the similarity-weighted null, simulated-paper null models have substantially lower fill rates than real papers, with dimension 1 and 2 rates virtually zero across disciplines.In dimension 0, rates are only a fraction of real-paper rates.Source: Holes Filled by Simulated Papers · p. 31
  • For similarity-weighted simulations, lowering temperature from 2 to 0.05 brings fill rates toward real-paper levels, while T=0.01 lowers rates across dimensions.Rates approach empirical levels only when concept arrangements sufficiently resemble real data; at T=0.01, the model is too constrained to fill holes.Source: Holes Filled by Simulated Papers · p. 31

Papers That Fill Holes

  • Across Dimensions 0, 1 and 2, enrichment rises in empirical and natural sciences (1.47 →1.89 →2.62), while formal and design fields decline (1.20 →1.08 →0.89), crossing below unity by Dimension 2.Source: Papers That Fill Holes · p. 31
  • Content surprise rises from Dimension 0 to 1 in natural sciences with data, mathematics, and philosophy, but declines with dimension in economics and history.Thus, larger-gap fillers often combine more surprising contents, even in fields where they gain no additional citations.Source: Papers That Fill Holes · p. 32
  • Context surprise declines from Dimension 0 to 1 in chemistry, engineering, medicine, and physics, while the overall dimension trend is carried by geology and philosophy.Fields marked mildly novel by the content variant are marked conventional by the context variant, and vice versa.Source: Papers That Fill Holes · p. 32

Prescience of hole-filling papers

Death-value analysis of hole-filling papers

Analysis

  • Citation carries by far the strongest death structure of any channel, while pooled citation enrichment seems to depend little on death value because relationships vary by discipline: 9 of 19 are positive and 10 negative.Source: Analysis · p. 36
  • In H0, citation-enrichment slopes average +0.50 for empirical disciplines versus -0.56 for formal/theoretical disciplines (difference +1.06); in H1, they average +0.52 versus -0.25 (difference +0.77).In H0, all 8 empirical fields have positive slopes and 10 of 11 formal fields have negative slopes; in H1, the counts are 6 of 8 and 10 of 11, respectively.Source: Analysis · pp. 36–37
  • In one group of disciplines, papers closing larger gaps accumulate citations; in the other, the citation advantage is greatest for the smallest gaps and decays as gaps grow.Discipline-level values are reported for H0 and H1 in Tables S4 and S5.Source: Analysis · p. 37
  • The citation advantage peaks at large gaps in empirical fields, while the interaction coefficients average -0.02 versus -0.15 in H0 and +0.11 versus -0.22 in H1 for empirical versus formal fields.The two-dimensional model reproduces this division; its interaction coefficient locates the most enriched part of the surface.Source: Analysis · p. 37
  • In H0, death coefficients are negative in 7 of 19 disciplines, positive in 4, and within 0.05 of zero in the remaining 8; Art’s coefficient is -0.41.Source: Analysis · p. 37
  • In H0, content-surprise and citation-count death slopes order the 19 disciplines nearly identically, but only 11 of 19 disciplines share a coefficient sign across the metrics.Most content-surprise coefficients lie very close to zero, helping explain why the sign agreement is close to random chance.Source: Analysis · p. 38
  • In H1, content-surprise death coefficients are negative in 9 disciplines, positive in 2, and close to zero in 8; the median coefficient declines from 1.21 to 1.13 across the death axis.The interaction coefficients are dominated by History +2.59 and Art +1.91, far outside the range of the other 17 disciplines.Source: Analysis · p. 38
  • In H0, the death coefficient is negative in 10 of 19 disciplines, and the interaction coefficient is negative; median enrichment declines from 1.53 in the lower death-axis half to 1.39 in the upper half.Source: Analysis · pp. 38–39
  • In H1, context-surprise death coefficients are negative in 12 disciplines; within the top decile they are negative in 13 and positive in 2, and the slope and coefficient signs agree in all 19.The median discipline’s enrichment falls from 1.59 to 1.33 between the death-axis halves.Source: Analysis · p. 39
  • Content prescience shows modest H0 death dependence: no discipline’s filling-paper advantage grows appreciably with death value, and median enrichment shifts only from 1.11 to 1.09.The death coefficient is negative in 7 disciplines, near zero in 11, and positive in one; top-decile dependence is negative in 12, negligible in 6, and positive in one.Source: Analysis · p. 40
  • Context prescience arranges disciplines much like context surprise in H0, but this arrangement disappears in H1.Source: Analysis · p. 40
  • In H0, death coefficients are negative in 8 of 19 disciplines, positive in 4, and near zero in 7; top-decile death dependence is negative in 13, positive in 4, and negligible in 2.Source: Analysis · pp. 40–41
  • Dependence is negative in 11 disciplines and positive in 5; the majority is negative in both dimensions, yet the dimensions do not agree about which disciplines are negative.Source: Analysis · p. 41
  • The discipline pattern in citation death slopes recurs more weakly in H1 than H0, and both prescience channels have smaller slope magnitudes than citation.Source: Analysis · pp. 46–47
  • For biology, chemistry, computer science, engineering, medicine, and physics, dimension-2 hole counts were extrapolated from power laws because persistent homology was computationally infeasible.Source: Figure S1 · p. 53, Analysis · p. 53
  • All fields show more highly cited papers bridging zeroeth-order gaps and tiling first-order holes, but citation likelihood rises with hole dimensionality only in empirical fields and falls in formal and design fields.The comparison covers papers in the top 25% of citations.Source: Figure S3 · p. 58, Analysis · p. 58
  • Synthetic papers preserve each real paper’s concept count and concept popularity while randomizing which concepts occur together.Concepts are sampled without replacement according to the empirical frequency distribution across papers in the discipline.Source: Analysis · p. 62
  • A concept-identity permutation preserves the holes and real papers while breaking the alignment between holes and their originating concepts.A hole is counted as filled when a real paper contains the relabeled concept set; Figure S8 reports filled-hole counts by discipline and homology dimension.Source: Analysis · p. 63, Figure S8 · p. 63
  • Semantic-similarity simulations replace each paper’s concepts with an equally sized set chosen by cosine similarity, with temperature controlling the strength of that preference.As temperature approaches zero, selection becomes strongly homophilous; at large temperatures it approaches uniform randomness. Figures S9–S14 show filled-hole counts by discipline and homology dimension for different temperatures.Source: Analysis · p. 64, Figure S9 · p. 64
  • Figure S15 presents hole-filler properties separately by discipline, with the cited page showing art.The supplied passages do not provide the plotted values or discipline-level comparisons.Source: Figure S15 · p. 70
  • Figure S16 measures the share of works filling 0-, 1-, and 2-dimensional holes above content or context prescience thresholds, alongside ratios to all works.The ratios divide the share of hole-filling works above each threshold by the share of all works at that threshold.Source: Figure S16 · p. 80, Analysis · p. 80
  • Figure S17 presents the same prescience analyses separately by discipline, with one discipline per page.The supplied pages include art, biology, economics, physics, and political science, among others.Source: Figure S17 · p. 81, Analysis · pp. 82, 86, 96, 97
  • Figure S18 plots the correspondence between death percentiles and their values for dimensions 0, 1, and 2.Source: Figure S18 · p. 99, Analysis · p. 99
  • Figure S19 shows pooled 3D surfaces of hole-filling paper proportions by homology dimension, ranking-metric percentile, and hole death percentile.The surfaces pool 19 disciplines, weighted by paper count.Source: Figure S19 · p. 100, Analysis · p. 100
  • Figure S20 plots ratios of hole-filling proportions to baseline proportions across metric and death percentiles, pooled across disciplines.Source: Figure S20 · p. 101, Analysis · p. 101
  • Figure S21 plots citation-surprise metrics by discipline, comparing hole-filling paper proportions across metric and death percentiles with the baseline.The figure includes citation, content-surprise, and context-surprise panels; the art page is identified in the supplied caption text.Source: Figure S21 · p. 102, Analysis · p. 102
  • Figure S22 shows pooled surface plots of hole-filling paper proportions across content- and context-prescience percentiles.These plots pool across all disciplines.Source: Figure S22 · p. 120, Analysis · p. 120
  • Figure S23 plots, pooled across all disciplines, ratios of the proportion of hole-filling papers by metric percentile and death percentile to the baseline proportion.The black contour line at z = 1 separates regions above and below 1.Source: Figure S23 · p. 121, Analysis · p. 121
  • Figure S24 presents discipline-specific ratio plots for content- and context-prescience, comparing above-percentile hole-filling proportions with the baseline.The supplied caption identifies an art page among the one-discipline-per-page plots.Source: Figure S24 · p. 122, Analysis · p. 122
  • Figure S24 continues plots of hole-filling-paper ratios by metric and death percentile against baseline proportions, organized by discipline for content and context prescience.The supplied passages provide figure-caption fragments and labels, but no plotted values or results.Source: Analysis · pp. 132–133

Related papers

Loading 2608.28822v1…