PaperlayerView exact revision on arXiv

Source-linked AI summary

A narrowing of AI research?

Joel Klinger, Juan Mateos-Garcia, Konstantinos Stathoulopoulos

arXiv:2009.10385v4cs.CY

TL;DR

The paper asks whether AI research diversity is narrowing as deep learning and private-sector participation grow. It examines arXiv research and finds stagnating diversity, narrower private-sector focus, and greater citation influence for private research.

  • Problem

    The paper asks whether AI’s technological diversity needs support to avoid premature lock-in to a potentially flawed dominant design.

    Source: §1 · p. 2, §2.3 · p. 7

  • Method

    The paper examines AI research diversity over time and compares private-sector research with other participants, including its citations and collaborations.

    Source: §1 · p. 2, §5.1 · p. 33

  • Results

    AI thematic diversity has stagnated, private-sector research is narrower, and it receives more citations even after topical focus is controlled.

    Source: §5.1 · p. 33

  • Takeaways & Limitations

    The findings provide a rationale for policies to preserve thematic diversity, although evidence about the mechanisms is insufficient to confidently recommend specific interventions.

    Source: §5.3 · p. 35

    The analysis does not establish the mechanisms or causal impact through which private-sector participation may shape AI research.

    Source: §5.2 · p. 34

Abstract

from arXiv · show

The arrival of deep learning techniques able to infer patterns from large datasets has dramatically improved the performance of Artificial Intelligence (AI) systems. Deep learning's rapid development and adoption, in great part led by large technology companies, has however created concerns about a premature narrowing in the technological trajectory of AI research despite its weaknesses, which include lack of robustness, high environmental costs, and potentially unfair outcomes. We seek to improve the evidence base with a semantic analysis of AI research in arXiv, a popular pre-prints database. We study the evolution of the thematic diversity of AI research, compare the thematic diversity of AI research in academia and the private sector and measure the influence of private companies in AI research through the citations they receive and their collaborations with other institutions. Our results suggest that diversity in AI research has stagnated in recent years, and that AI research involving the private sector tends to be less diverse and more influential than research in academia. We also find that private sector AI researchers tend to specialise in data-hungry and computationally intensive deep learning methods at the expense of research involving other AI methods, research that considers the societal and ethical implications of AI, and applications in sectors like health. Our results provide a rationale for policy action to prevent a premature narrowing of AI research that could constrain its societal benefits, but we note the informational, incentive and scale hurdles standing in the way of such interventions.

1 Introduction

The paper asks whether deep learning’s rise is narrowing AI research despite concerns about brittleness, environmental costs, and unfair outcomes. It examines thematic diversity over time and compares private-sector research with the rest of the field.

1 Introduction

  • Technological lock-in can entrench an inferior design, motivating policies that preserve diversity while the benefits and risks of alternatives remain uncertain.The introduction uses the combustion engine and light-water nuclear reactors as examples of designs that became dominant despite drawbacks.Source: §1 · p. 2
  • Deep learning’s commercial success and limitations frame concerns that the AI research trajectory may narrow prematurely.The cited concerns include brittle systems, environmental costs, and discrimination risks from training on biased data.Source: §1 · pp. 2–3
  • The study analyzes arXiv AI research to track thematic diversity, compare private firms with other contributors, and assess company influence through citations and collaborations.It estimates the corpus’s thematic composition with a topic model and calculates diversity using multiple metrics.Source: §1 · p. 3

2 Literature Review

The review frames AI’s current trajectory as one possible technological design among alternatives, and lays out why diversity may matter, how it can be narrowed, and what role firms may play.

2 Literature Review

  • The review situates its argument in four claims: technological designs can coexist, diversity may be desirable, economic forces can narrow it, and firms may have weaker incentives to preserve it.These claims are connected to AI’s history, dominant trajectory, and the influence of large technology companies.Source: §2 · p. 3

2.1 Technological directionality and AI’s technological trajectory

  • AI has moved from symbolic methods and expert systems to machine learning and deep learning; system designs shape inputs, costs, outputs, and deployment opportunities.Deep learning’s recent gains rely on learning patterns from large unstructured datasets.Source: §2.1 · pp. 3–4

2.2 The value of technological diversity

  • Technological diversity is presented as supporting resilience, inclusion, pluralism, and creative recombination, with deep learning’s weaknesses and sector-specific needs sharpening the case.The review cites generalization limits, computational costs, privacy and fairness risks, and the suitability of health applications as concerns.Source: §2.2 · pp. 5–6

2.3 Drivers of technological homogenisation

  • Scale economies and network effects can reinforce a dominant design; AI’s hardware lottery and competitive races may further favor established, short-term research lines.The review describes GPUs as an enabling complement for deep learning and notes that races may encourage shortsighted choices.Source: §2.3 · pp. 6–7
  • Because technological diversity has public-good features, individual researchers may have incentives to work on dominant technologies rather than preserve alternatives.The reviewed models describe overinvestment in perceived high-return areas and insufficient exploration of unresolved areas.Source: §2.3 · p. 7

2.4 The private sector and AI’s technological diversity

  • Commercial firms may favor readily deployable technologies and efficiency, while prior work raises concerns that corporate resources and collaborations skew research agendas toward dominant designs.The review also notes firms’ active role in AI research and open innovation.Source: §2.4 · pp. 7–8

3.1 Research questions

The paper sets out to provide descriptive empirical evidence on AI’s thematic diversity and the private sector’s role in the research ecosystem.

3.1 Research questions

  • The first research question asks whether AI research is becoming thematically narrower.The paper frames this as a possible decline in diversity and risk of lock-in to a dominant design.Source: §3.1 · pp. 8–9
  • The second research question compares the thematic diversity of private-sector organizations with that of academia and the public sector.Source: §3.1 · p. 9
  • The third research question asks how influential private-sector AI research is.Source: §3.1 · p. 9

3.2 Data sources

The study builds a dataset linking arXiv research records with author affiliations and institution types, while noting coverage and disclosure limits relevant to institutional comparisons.

3.2 Data sources

  • The authors create a dataset combining arXiv, Microsoft Academic Graph, and the Global Research Identifier.Source: §3.2 · p. 9

3.2.1 arXiv

  • The arXiv corpus contains 1.84 million articles published from April 1986 through August 2020, with article metadata, abstracts, and citation counts.Source: §3.2.1 · p. 9

3.2.2 Microsoft Academic Graph (MAG)

  • Microsoft Academic Graph provides author affiliations at publication time, identified by querying article titles.Source: §3.2.2 · p. 9

3.2.3 Global Research Identifier (GRID)

  • Fuzzy matching MAG institutions to GRID produced just over one million articles and 2.45 million article-institute pairs; duplicate organizations were collapsed and DeepMind and OpenAI records corrected.The analysis assumes DeepMind papers do not involve Google researchers, although the authors note this is not always true.Source: §3.2.3 · pp. 9–10

3.2.4 Observations about potential biases

  • arXiv is unreviewed and may omit undisclosed company results; matching may miss smaller institutions, with ambiguous diversity effects, though full-corpus trends avoid that matching gap.The authors expect undisclosed company results to include more downstream or applied findings.Source: §3.2.4 · pp. 10–11

3.3 Methods

The methods identify AI papers beyond core AI categories, infer topic distributions from abstracts, and measure thematic diversity using three complementary metrics. The authors also test robustness to alternative identification and topic-modeling choices.

3.3.1 Procedure to identify AI articles

  • The AI corpus expands beyond four core categories by identifying papers outside them through unusually frequent AI-technique terms.The core categories are cs.AI, cs.NE, cs.LG, and stat.ML; the identification procedure expands terms found in those categories to locate applications elsewhere.Source: §3.3.1 · p. 11
  • Because AI identification uses heuristically selected parameters, the authors also test an alternative strategy that incorporates citation information.Its impact is assessed in Subsection 4.1.3.Source: §3.3.1 · p. 11

3.3.2 Topic modelling

  • TopSBM estimates topics by decomposing a network of words co-occurring in article abstracts into communities using a stochastic block model.The resulting topic and article topic mixes are treated as proxies for paper themes and aggregated thematic composition.Source: §3.3.2 · p. 12
  • TopSBM reduces reliance on manually selecting topic counts and makes weaker assumptions than LDA, but the authors check results with LDA because they rely on one algorithm.The model automatically estimates the number of topics; the alternative-model robustness analysis is reported in Subsection 4.1.3.Source: §3.3.2 · p. 12

3.3.3 Metrics of diversity

  • The diversity measures capture different dimensions: balance reflects topic concentration, Weitzman reflects topic distances, and Rao-Stirling combines balance with distance.The authors use the measures in parallel to assess robustness to different definitions and operationalisations of diversity.Source: §3.3.3 · p. 13
  • Unlike balance and Rao-Stirling, Weitzman does not account for topic shares, so a long tail of distant minority topics can yield high diversity despite concentration.The topic distances are based on co-occurrences in articles; the authors do not add Weitzman's separate analysis of preservation costs.Source: §3.3.3 · pp. 13–14

4.1 Evolution of AI research and its thematic diversity

The balance and Rao-Stirling measures show AI research diversity rising after 2012 and stagnating from 2017, a pattern generally retained in robustness tests. Meanwhile, AI research grew rapidly and its topical mix shifted toward computer vision and language, while symbolic and non-deep-learning areas declined in relative importance.

4.1.1 AI research trends

  • AI research grew more recently and rapidly than non-AI arXiv categories, reaching 20% of uploads in the first half of 2020.60% of the corpus's AI papers were published after 2018.Source: §4.1.1 · pp. 14–15, Figure 1 · p. 15
  • Figure 2 assigns topics to the arXiv category where each is most represented, using a topic-occurrence threshold above 0.1 per paper.The authors note that this categorisation supports interpretation of research evolution, not the diversity analysis itself.Source: §4.1.1 · p. 15
  • The topical mix shifted toward computer vision and language, while symbolic-focused cs.AI and non-deep-learning statistical machine learning topics declined in relative importance.The passage also reports growth in cs.CR, cs.CY, cs.IR, and robotics topics.Source: §4.1.1 · p. 16, Figure 2 · p. 16

4.1.2 Evolution of thematic diversity

  • The balance metric, tracking activity distribution over topics, and Rao-Stirling, weighting balance by distance between topics, show similar trends: diversity strongly increased after 2012, then stagnated from 2017 onwards.Source: §4.1.2 · p. 18

4.1.3 Robustness tests

  • The diversity pattern generally persists across alternative AI-paper identification, LDA topic models, and a temporally balanced corpus, though some Weitzman results differ.The authors report an initial increase in the mid-2010s followed by stagnation and, in some cases, decline.Source: §4.1.3 · p. 19

4.2 Thematic diversity in the private sector

Private-company involvement in arXiv AI research is substantial and associated with lower thematic diversity; company and academic research also differ in their topical profiles.

4.2.1 Private participation in AI research

  • Private companies account for around 10% of AI research participations, a share ten times higher than their participation outside AI.Academic institutions remain the most active organisation type in the corpus.Source: §4.2.1 · p. 20, Figure 5 · p. 21
  • Private-sector participation in AI research has almost doubled over the last decade, while healthcare and nonprofit participation has remained stable and below their outside-AI levels.Government institutions played a larger role before 2010, when AI research volumes were very low.Source: §4.2.1 · pp. 20–21, Figure 6 · p. 21
  • The 15 most active companies' share of AI papers peaked around 25% in 2015–2019 and declined after 2019 as arXiv AI research grew faster.In the first months of 2020, almost two in ten AI papers involved one of these companies.Source: §4.2.1 · p. 22, Figure 7 · p. 22

4.2.2 Comparing the thematic diversity of private and public AI research

  • Thematic diversity in private AI research is consistently lower than in non-private research across all tested metrics and parameter sets.The comparison uses repeated random samples of 1,000 articles from each group.Source: §4.2.2 · p. 23, Figure 8 · p. 23

4.2.3 Modelling thematic diversity at the organisation level

  • Company status is negatively associated with balance and Rao-Stirling diversity; Weitzman associations are generally negative but weaker and once positive.The negative association strengthens and becomes significant when organisation fixed effects are included; larger publication volume is positively associated with diversity across metrics.Source: §4.2.3 · p. 24, Table 5 · p. 25

4.2.4 The thematic profile of private sector companies in AI research

  • Private research specialises more in language, computing, information retrieval and vision, while academics specialise more in health applications and AI ethics and law.Companies are also less focused on symbolic AI, neural and evolutionary computing, statistical machine learning, and computers and society.Source: §4.2.4 · p. 26, Figure 10 · p. 29

4.3 The influence of private sector organisations in AI research

The paper examines private-sector influence through citations and collaborations. Company-involving research receives more citations, and private companies collaborate with influential institutions whose profiles are relatively narrow.

4.3.1 Citations to private AI research

  • Articles involving private companies receive around twice as many citations, and their citation advantage remains after adjustment for publication year, institution count and topic composition.Including topic composition reduces the company coefficient, consistent with private companies specialising in more popular topics.Source: §4.3.1 · pp. 30–32, Table 7 · p. 30, Table 8 · p. 31
  • The authors conclude that private-company AI research is particularly influential and could steer the field toward research profiles resembling its narrower company profiles.Mean citation-count differences have dwindled in recent years, possibly because newer papers have had less time to accrue citations.Source: §4.3.1 · pp. 30, 32

4.3.2 Private sector collaboration patterns

  • Sixty percent of papers involving private companies also include an educational institution, and their top collaborators are influential US institutions with relatively narrow research profiles.Mean citation counts decrease monotonically down the rankings of institutions collaborating with private AI researchers.Source: §4.3.2 · pp. 32–33, Table 9 · p. 32

5 Conclusion

AI research diversity has stagnated and may be declining, while private-sector research is more narrowly focused and influential. The discussion weighs the value and limits of diversity and outlines policy options alongside substantial implementation hurdles.

5.1 Discussion of results

  • AI research diversity has stagnated and may be declining; private-sector research is narrower, receives more citations controlling for topical focus, and collaborates actively with prestigious universities.The authors speculate that these collaborations could indirectly influence follow-on research, but say the mechanism requires further study.Source: §5.1 · p. 33
  • Private-sector research concentrates on digital-economy techniques and deep-learning infrastructure, with less emphasis on health applications and ethical and legal considerations.Elite US universities that collaborate actively with firms also have comparatively narrow profiles; smaller academic institutions may help preserve diversity.Source: §5.1 · p. 33

5.2 Limitations and issues for further research

  • The value of thematic diversity is assumed, but some diversity may reflect scholarly inertia, and relative metrics may undercount ideas explored by a growing research community.The authors call for research to quantify diversity’s contributions to resilience, inclusion, and creativity, and propose valuing topics in light of minimum viable activity thresholds.Source: §5.2 · pp. 33–34
  • The descriptive analysis does not establish how organizations and technologies co-evolve or whether private-sector participation causally shapes AI research.The study uses one pre-print data source and semantic methods; the authors recommend testing other sources, metrics, and operationalizations.Source: §5.2 · p. 34

5.3 Policy implications

  • Proposed options include diversifying researchers’ backgrounds, supporting independent academic compute, broadening benchmarks and funding agendas, and backing AI applications in underserved domains.The authors caution that evidence about mechanisms is insufficient to confidently recommend specific interventions.Source: §5.3 · p. 35
  • Policy efforts face incentive, information, and scale hurdles, so the authors urge preserving public-interest AI R&D and patiently investing in alternative techniques and less-data-intensive applications.Current economic and geopolitical incentives favor state-of-the-art technologies; alternatives can be harder to assess, and public research budgets are small relative to large technology firms’ R&D budgets.Source: §5.3 · pp. 35–36

Technical annex

The annex describes how the authors expanded their arXiv AI corpus beyond core categories and tested its coverage using citation-linked papers. The validation found broad topic-distribution similarity but also potential omissions, prompting an alternative corpus operationalization for robustness tests.

Detailed procedure to identify AI papers outside of core categories

  • The expanded corpus includes papers outside core AI categories when their counts of category-specific salient terms exceed a threshold derived from inside- and outside-category frequencies.Salient vocabularies use frequent category-associated n-grams and semantically similar terms; the threshold includes a category-specific scaling factor.Source: Detailed procedure to identify AI… · pp. 40–41
  • The authors manually selected scaling factors to reduce false positives, especially where generic terms could misclassify non-AI research.The category vocabularies show deep-learning terms in cs.NE, cs.LG, and stat.ML, while cs.AI includes terms associated with symbolic approaches.Source: Detailed procedure to identify AI… · p. 41, Table 10 · p. 42, Table 11 · p. 43

An alternative strategy to identify AI papers

  • Most papers citing or cited by the sample were already in the AI corpus, with a higher in-corpus proportion among papers more influential in or influenced by it.The authors extracted 72,603 citations to the sample and 360,730 citations from it; the figure shows the proportion across citation-distribution positions.Source: An alternative strategy to identify AI… · p. 44, Figure 12 · p. 44
  • Mean topic weights were strongly correlated between the AI sample and external papers citing from it (Spearman ρ = 0.96) or citing to it (ρ = 0.92).Figure 13 compares average topic weights by article category, with topics ranked by average weight.Source: An alternative strategy to identify AI… · p. 45, Figure 13 · p. 45
  • Citation-linked papers citing the AI corpus overrepresented computer-vision and natural-language-processing topics, suggesting omitted applications and motivating an alternative AI-corpus definition in robustness tests.Papers cited by the AI corpus overrepresented physics and materials-science topics, consistent with AI research drawing on other disciplines.Source: An alternative strategy to identify AI… · p. 45

A network analysis of the composition of AI research

Topic co-occurrence networks compare AI research in 2013–2016 with 2019–2020. The later network is more densely connected, with deep-learning-related categories more prominent and network distances reduced.

A network analysis of the composition of AI research

  • The authors compare topic co-occurrence networks across two periods, using a 300-topic LDA model and a maximum spanning tree to retain the strongest connections in a connected display.Nodes represent topics, edges their co-occurrences, node size the number of topic occurrences, and color the arXiv category where a topic is most salient.Source: A network analysis of the composition… · pp. 45–46
  • The 2019–2020 topic network is more densely connected than the 2013–2016 network, with deep-learning-related categories more prominent and topic distances reduced.The earlier network had more disconnected components and, in its largest component, a longer diameter and average path length; the display uses a maximum spanning tree.Source: A network analysis of the composition… · p. 46, Figure 14 · p. 47, Table 13 · p. 48

Related papers

Loading 2009.10385v4…