PaperlayerView exact revision on arXiv

Source-linked AI summary

Large scientific teams are more, not less, disruptive

Junming Huang, Yu Xie

arXiv:2610.07329v2cs.DLphysics.soc-ph

TL;DR

The reported small-team disruption advantage is difficult to interpret because reference counts vary with team size and affect the standard disruption index. The study examines this bias, validates a reference-robust measure against expert-curated breakthroughs, and finds that disruption increases with team size.

  • Problem

    The reported small-team disruption advantage is difficult to interpret because larger teams cite more references and reference count strongly affects the standard disruption index.

    Source: The Small-Team Advantage as a… · p. 2

  • Method

    The study balances reference-count distributions within the standard index and evaluates a one-reference-at-a-time measure against six expert-curated breakthrough benchmarks.

    Source: Team Size and Scientific Disruption… · p. 7, Discussion and Conclusion · p. 10

  • Results

    Under 1D, disruption increases with team size across major scientific domains and publication decades, while 1D ranks recognized papers above background papers across all six benchmarks.

    Source: Discussion and Conclusion · pp. 10–11

  • Takeaways & Limitations

    For scientific publications, once the standard index’s mechanical dependence on reference count is addressed, larger teams are more, not less, disruptive.

    Source: Discussion and Conclusion · p. 11

    The observed positive team-size gradient is not causal: the design does not identify the mechanism or estimate the effect of adding an author.

    Source: Discussion and Conclusion · p. 12

Abstract

from arXiv · show

Modern science has moved decisively toward larger and more collaborative research teams. Yet an influential 2019 study reported that small teams are more disruptive than large teams, a conclusion difficult to reconcile with both the well-known trend toward increasing collaboration and the disproportionately large teams behind widely recognized disruptive research, such as work that has been awarded Nobel Prizes. Here we show that the reported disruptive advantage of small teams is largely an artifact of measurement rather than a fact about teams. Analyzing about 60 million scientific publications, we find that larger teams cite more references and that the standard disruption index declines mechanically as reference lists lengthen. Equalizing reference-count distributions across team sizes reverses the negative gradient within the index's own framework. Evaluating each paper against one reference at a time yields a reference-robust measure that distinguishes recognized breakthroughs from ordinary papers more sharply and more consistently than the standard disruption index across six independently curated benchmarks, including the Nobel Prize. Under this validated measurement, scientific disruption increases with team size across four scientific domains and in every decade from the 1970s to the 2010s. For scientific publications, we conclude that larger teams are more, not less, disruptive.

Introduction

The paper examines how reference-count bias affects measures of scientific disruption and uses a reference-robust measure to assess disruption by team size. It also considers whether that measure identifies recognized advances consistently across fields and decades.

Introduction

  • The claim that small teams disrupt while large teams develop (Wu et al., 2019) conflicts with the rise of collaboration and the disproportionate presence of larger teams among Nobel-winning papers.The paper argues that the reported small-team advantage does not hold once citation-practice differences are accounted for.Source: Introduction · p. 2

The Small-Team Advantage as a Methodological Artifact

Team Size and Scientific Disruption Revisited

  • Equalizing reference-count distributions reversed the standard index’s negative team-size gradient to positive at every citation window, but this is a counterfactual, not causal identification.The adjustment deletes references and changes the composition of reference sets and the index components.Source: Team Size and Scientific Disruption… · p. 7
  • Across six expert-curated benchmarks, 1D achieved AUCs of 0.94–0.98, versus 0.26–0.79 for the standard disruption index.The standard index fell below chance on two benchmarks, whereas 1D consistently distinguished recognized breakthroughs from background papers.Source: Figure 4 · p. 9, Team Size and Scientific Disruption… · p. 10
  • Under a measure that passes independent validity tests, scientific disruption in publications increases with team size across four major scientific domains, every decade from the 1950s to the 2010s, and every citation-window choice.Source: Team Size and Scientific Disruption… · p. 10

Discussion and Conclusion

  • The authors conclude that reference count explains more variation in the standard index than team size does, and that aligning reference counts reverses the apparent small-team advantage.Their reference-robust One-Reference Disruption Index removes the cumulative penalty from adding references.Source: Discussion and Conclusion · p. 10
  • Under uniform 1D, disruption increases with team size across scientific domains and publication decades, with the highest mean among the largest teams.The pattern appears in Health Sciences, Life Sciences, Physical Sciences, and Social Sciences, and in each decade from the 1950s to the 2010s.Source: Figure 5 · p. 11, Discussion and Conclusion · p. 11
  • Across all six benchmarks, 1D distinguishes recognized papers from background papers more clearly and consistently than the standard disruption index.On two benchmarks, the standard index ranks a recognized paper below a background paper more often than above it.Source: Discussion and Conclusion · p. 12
  • The positive team-size gradient is not a causal estimate, and 1D removes the mechanical reference-count penalty without eliminating all association with reference count.Reference balancing is diagnostic, not causal; unresolved mechanisms and possible systematic bibliographic-record errors limit interpretation.Source: Discussion and Conclusion · p. 12

Data and Methods

  • The analysis covers 47,134,871 focal papers published before 2020, each with at least one author, five references, and one recorded citation.Team size is defined as listed author count, and field analyses assign each paper to the top-level domain of its primary topic.Source: Data and Methods · pp. 12–13
  • A five-reference minimum was chosen as a compromise: it excludes 11.44% of eligible papers, compared with 25.38% under a ten-reference threshold.The cutoff aims to exclude short-reference non-research works without discarding as many substantive research articles as a higher threshold would.Source: Data and Methods · p. 13

Team size and reference count.

This section describes a decade-adjusted analysis of team size and reference count, a reference-balancing diagnostic, and validation against expert-curated breakthrough benchmarks.

Team size and reference count.

  • The decade-adjusted regression compares mean reference counts across ordered team-size groups; its 10+ group is top-coded, so the slope is descriptive rather than an effect per added author.Source: Team size and reference count · p. 14
  • For each team size above one, references were randomly deleted to match the single-author reference-count distribution; recomputed DI averages across 12 replicas serve as a measurement diagnostic, not causal identification.Source: Team size and reference count · p. 15
  • Validation compares standard DI and Monte Carlo 1D on the same positive and negative paper groups, using AUC to measure discrimination across six benchmark collections.Source: Team size and reference count · pp. 15–16
  • Reference count was not a consistent benchmark discriminator: its AUC ranged from 0.31 to 0.65, below 0.5 for Nobel Prize and Turing Award papers, so it cannot explain 1D’s consistently high discrimination.Source: Team size and reference count · p. 16

Data availability

Code availability

A Eligibility and coverage diagnostics

The eligibility diagnostic shows that citation-based inclusion is selective by team size, so the main curves describe papers in the observed citation network.

B Team size and reference count

The decade-cell analysis finds a positive pooled descriptive slope, while cautioning that this does not establish a positive association in every decade.

C Mechanical dependence of the standard disruption index on reference count

The standard disruption index declines with reference count because its normalized components decline at different rates, with the score approaching zero at high counts.

  • More predecessors create more opportunities for following papers to cite a focal paper’s references; later papers citing both the focal paper and its predecessors count in NB rather than NF.Source: C Mechanical dependence of the… · p. 22
  • The mechanism compares reference counts across focal papers and does not assume that one paper’s longer reference list contains another paper’s shorter list.Source: C Mechanical dependence of the… · p. 22
  • NF/(NF + NB + NR) declines faster than NB/(NF + NB + NR); they cross near 11–12 references, where mean DI changes sign, then DI approaches a near-zero floor at high counts.Source: C Mechanical dependence of the… · p. 23

C.1 Degree-preserving randomized citation networks

The randomized-network test asks whether the DI–reference-count decline requires substantive differences in disruption. Most of the observed decline persists after citation partners are randomized.

C.1 Degree-preserving randomized citation networks

  • Stub permutation changes citation partners while targeting observed in-degree and out-degree within each of 10,000 community-block shards.The construction preserves the community-block edge structure and is not a global configuration-model draw.Source: §C.1 · pp. 23–24
  • Each shard may differ from its original edge count by up to 10 links, so the randomized network is degree-preserving only within a small numerical tolerance.The reported shuffling error is below 0.005%; the aggregate discrepancy is described as negligible.Source: §C.1 · p. 24
  • The negative DI–reference-count association is also visible within each decade from the 1970s through the 2010s.Source: Figure S3 · p. 25
  • About 85% of the observed DI decline with reference count remains after citation partners are randomized, showing the decline does not require substantive disruption differences.The decline is approximately 0.0165 in the observed network and 0.0141 in randomized networks.Source: §C.1 · pp. 24–25

D Reference-balancing algorithm and diagnostics

The reference-balancing design thins larger-team papers to the single-author reference-count distribution using deletion only. It aligns distributions at the binned level, while deletion itself changes DI scores mechanically.

D Reference-balancing algorithm and diagnostics

  • For each larger-team group, a monotone coupling assigns retained reference-count bins no greater than the original bins, targeting the single-author distribution.References are thinned rather than added; counts in the pooled 50+ tail are not further thinned within that bin.Source: D Reference-balancing algorithm and… · p. 25
  • The standard DI’s reference-count decline arises as the NF share falls faster than the NB share, while NR is much larger than either component.In randomized networks, NB remains near zero while NF share still declines.Source: Figure S4 · p. 26
  • The coupling maps each source distribution to the single-author target in expectation, with references selected uniformly without replacement after the retained count is chosen.Sequential hypergeometric allocation across storage shards is equivalent to selecting a uniform subset from the complete linked reference list.Source: D Reference-balancing algorithm and… · pp. 25–26
  • Absolute DI levels before and after balancing are not directly comparable because deleting references changes the DI calculation mechanically.Source: D Reference-balancing algorithm and… · p. 26
  • Shorter-window supplementary samples are cumulative through later decades, not independent cohorts.Source: D Reference-balancing algorithm and… · p. 26

E Exact 1D and Monte Carlo convergence

The paper estimates 1D by sampling reference-specific pairwise scores and averaging across draws. Its validation uses the paper-level mean before calculating AUC, rather than averaging single-draw AUCs.

E Exact 1D and Monte Carlo convergence

F Reference count, citation windows, and 1D robustness

The robustness analyses examine 1D across reference counts, team sizes, citation windows, domains, and publication decades. They also clarify that removing the cumulative penalty does not make 1D independent of reference count.

F Reference count, citation windows, and 1D robustness

  • Mean 1D increases across much of the observed reference-count range, so the measure removes the mechanical cumulative penalty but is not statistically independent of reference count.The association may reflect differences in the kinds of relationships contained in longer bibliographies.Source: F Reference count, citation windows,… · p. 28
  • The window figures compare mean DI before and after balancing and mean 1D across pooled samples, domains, and available decades.Longer windows require complete follow-up and therefore restrict analysis to older focal-paper cohorts; comparisons must distinguish follow-up from cohort composition.Source: F Reference count, citation windows,… · p. 29, Figure S7 · p. 30, Figure S9 · p. 32
  • At 30 draws, the average Monte Carlo standard error is 1.7 × 10^-5, about three orders of magnitude below the rise from team size 1 to 10+.The reported rise is 190 × 10^-4.Source: Figure S8 · p. 31
  • The 30-draw estimate has reliability ρ30 = 0.948, and its correlation with the paper-level mean is √ρ30.Source: F Reference count, citation windows,… · p. 31

G Benchmark construction, attrition, and paired validation

The paired validation uses expert-curated breakthrough collections mapped to OpenAlex and a common eligible population. Reference count alone discriminates inconsistently across collections; the unrestricted DI implementation closely matches built-in DI.

Benchmark construction

  • The Lasker and Test of Time collections were assembled from official award lists, then deduplicated and mapped to OpenAlex works using bibliographic metadata and adjudication of ambiguous matches.Test of Time sources cover 12 computer science conference series and several award types.Source: G Benchmark construction, attrition,… · p. 33
  • Table S6 reports benchmark source-list and final eligible-positive counts; collections overlap, with 1,399 unique papers among 1,469 eligible collection entries.The common eligible population contains 41,152,454 papers.Source: G Benchmark construction, attrition,… · p. 33, Table S6 · p. 33

Attrition and paired population

  • Of 470 ineligible source records, 267 were absent from the OpenAlex citation network; 140 had fewer than five linked references, 56 lacked a SciSciNet built-in score, six were post-2019, and one had no recorded citation.Only 38 Turing Award papers remain eligible, so that collection’s AUC warrants caution.Source: G Benchmark construction, attrition,… · p. 33
  • The paired analysis uses 41,152,454 papers meeting its eligibility criteria: both scores, at least five references, at least one citation, and publication before 2020.The exclusion criteria overlap, and no eligible paper lacks 1D.Source: G Benchmark construction, attrition,… · p. 34

Paired validation

  • Reference count’s AUC is below 0.5 for Nobel Prize and Turing papers, near 0.5 for two collections, and above 0.5 for Lasker and Science Breakthroughs papers.Its simple average is approximately 0.50, but collection-level heterogeneity is more informative.Source: G Benchmark construction, attrition,… · p. 34
  • The six recognized-work collections do not uniquely define disruption, because recognition may also reflect influence, importance, or award conventions.Strictly increasing or decreasing transformations of reference count cannot produce consistently high discrimination across all six collections.Source: G Benchmark construction, attrition,… · p. 34
  • The unrestricted DI implementation’s AUC differs from built-in DI by no more than 0.030 across six benchmarks.This concordance supports using it for mechanism analyses while retaining the independently produced built-in score as the main outcome.Source: G Benchmark construction, attrition,… · p. 34
Loading 2610.07329v2…