The Role of Computing Resources in Publishing Foundation Model Research

1 Introduction

• Foundation models have advanced across modalities and are widely used as flexible backbones for innovation in other fields.The paper describes strong compositional performance and generalization in NLP and computer vision.Source: §1 · p. 1
• Research costs are difficult to quantify because resource distribution is not disclosed uniformly, making hardware purchases or rentals a more concrete cost measure.GPUs are especially salient because they are available in fixed quantities and face intense purchasing competition.Source: §1 · pp. 1–2
• The study benchmarks GPU counts and teraflops against more than 34,828 accepted papers from 2022 to 2024.It focuses on selective AI/ML computer-science conferences.Source: §1 · p. 2
• Greater GPU access is associated with higher acceptance rates across eight conference venues and higher citation counts, although other factors may influence the relationship.The authors identify institutional resources, reputation, and researcher networks as possible influences.Source: §1 · p. 2

2.1 LLM-extracted Publication Data

• The authors collected structured publication and author data through conference APIs, then used GPT-4o mini to extract senior-author affiliation, GPU use, dataset descriptions, and funding from PDFs.They attributed affiliation to the senior author’s first-listed institution and country, and classified papers by domain, phase, and method.Source: §2.1 · p. 2
• Among 34,828 papers collected from eight conferences, the study identified 6,517 FM-related papers and recruited 229 first authors representing 312 papers for a survey.Survey participants supplied self-reported resource information when it was absent from publications.Source: §2.1 · p. 2
• GPT-4o mini extracted GPU counts from 116 papers, a 59.7% gap against author reports; missing rates were 48.3% for GPU type and 88.6% for GPU hours.Two FM researchers independently checked 312 papers; authors supplied self-reports for 288 papers’ GPU counts, 292 papers’ GPU types, and 281 papers’ GPU hours.Source: §2.1 · p. 2
• GPU disclosures were sparse: 16.51% of papers reported GPU quantity, 24.22% GPU type, and 12.86% inference time.The authors say this inconsistent documentation prevents building structured data to understand and forecast the FM computing landscape; conferences with compute disclosure requirements tend to have higher GPU-specification reporting rates.Source: Figure 1 · p. 3

2.2 Self-Reported Author Survey Data

• The survey included 229 researchers who were first or co-first authors of publicly available papers, representing 118 institutions.The sample included 267 academia first authors and 36 industry first authors.Source: §2.2 · p. 4
• Authors reported more GPU use than appeared in published papers for 46.4% of the 140 papers with both sources; reports matched for 38.6% and were lower for 15.0%.Respondents said omitted usage could include experiments not reported in the final paper, or configurations beyond the setup supporting the largest experiment set.Source: §2.2 · p. 4

3.1 Foundation Model Research Is Growing in Count and Scope

• FM papers rose from 2.07% of accepted papers in 2022 to 10.29% in 2023 and 34.64% in 2024 across eight conferences.Inference-related work grew notably; algorithmic and empirical studies outpaced dataset/benchmark and toolkit categories.Source: §3.1 · p. 5
• Most projects used 1 to 8 GPUs, commonly 1 to 4, and GPU-use distributions remained steady despite the growth in FM research.The authors note that continued monitoring is warranted, especially as GPU purchase latency increases.Source: §3.1 · p. 5

3.2 Contributions from Both Industry and Academic Institutions

• Academic senior authors contributed more papers overall, but Google and Microsoft had the highest single-entity counts, followed by Tsinghua University, Meta, and Stanford.The study counts 4,851 papers from 611 academia institutions and 1,425 from 163 industry institutions, with 239 papers affiliated with other entities.Source: §3.2 · p. 5

3.3 FM Research is Led by Open-Weight Models

• Open-weight LLaMA models were used most frequently; the authors note that open weights support fine-tuning, domain adaptation, and benchmarking in ways proprietary models often do not.Proprietary models such as GPT also had a significant presence.Source: §3.3 · p. 5

3.4 GPUs Usage in FM Research

• Pretraining studies reported significantly more GPUs than post-training or inference studies (Mann-Whitney U test, p < 0.001); other GPU and TFLOP differences across categories were not statistically significant.The reported nonsignificant variations included lower median use in security-related research and higher use in toolkit construction.Source: §3.4 · p. 5
• Algorithmic development accounted for 47.4% of papers and empirical evaluation for 31.7%, while only 1.5% addressed toolkits or implementation infrastructure.A majority, 86.4%, focused on NLP.Source: §3.4 · p. 5
• Reported GPU counts commonly align with powers of two, such as 1, 2, 4, 8, and 16.The authors suggest this may reflect purchasing norms for GPU clusters.Source: §3.4 · p. 5

A Proportion of FM Papers Over Time

• The excerpt lists 31%, 32%, 34%, 29%, and 28%, alongside Total, COLM, ICLR, ICML, NeurIPS, NAACL, ACL, EACL, and EMNLP.The passage does not identify which percentage corresponds to which label.Source: A Proportion of FM Papers Over Time · p. 6

• Phase coverage is organized over time as inference, post-trained, and pre-trained work.Source: B Evolution of FM papers over phases,… · p. 6
• Domain coverage is organized over time across CV, data mining, explainable AI, NLP, other areas, and security.Source: B Evolution of FM papers over phases,… · p. 6
• Method coverage is organized over time across algorithms, data/benchmarks, empirical studies, and toolkits.Source: B Evolution of FM papers over phases,… · p. 6

• Four displayed series are each labeled Pub. and Survey.Source: C Temporal Evolution of GPU Model… · p. 6

Proportion (%)

• 48.7% of papers address inference, 36.1% post-trained analysis or adaptation, and 13.3% the pre-trained stage.Source: Proportion (%) · p. 6

3.5 Funding Sources for FM Research

• Government funding appears in 848 papers (85.5%), corporations in 291 (29.3%), and foundations in 102 (10.3%).GDP per capita is not consistently correlated with funded-paper counts across countries.Source: §3.5 · p. 6

A Senior Author Affiliation A₁

• The figure labels a panel as paper count by LLM use.Source: A Senior Author Affiliation A₁ · p. 7

3.6 Average Effort Required

• TFLOPs show a clearer positive relationship with paper output, particularly in industry, while GPU-count trends are inconsistent across academia and industry.The text identifies high-throughput compute infrastructure, rather than GPU quantity alone, as more decisive for sustained FM research output.Source: §3.6 · p. 7, Figure 4 · p. 8
• Higher-compute institutions tend to produce more-cited work, with TFLOPs more strongly associated with citations than GPU count.High-citation papers also occur at less compute-intensive institutions, so influence is not entirely dependent on resource scale.Source: §3.6 · p. 8, Figure 4 · p. 8
• 40.13 is the average citation count for FM papers (median: 8.00; standard deviation: 216.02).ICML has the highest average citation count among the ML conferences at 78.0 (SD: 312.0).Source: §3.6 · p. 8

3.7 Accepted and Rejected Papers Resource Usage

• Rejected ICLR papers averaged lower GPU usage, fewer TFLOPs, and smaller author teams than accepted papers, but their distributions were very similar.The authors note that ICLR is the only studied conference releasing rejected and withdrawn papers, limiting broader generalization.Source: §3.7 · p. 9

4 Discussion

• The authors say more GPUs do not inherently yield greater research impact, while unchecked computational expansion raises environmental concerns and compute access remains concentrated.They warn that China and the United States disproportionately dominate FM research.Source: §4 · p. 9
• Multiple GPUs used for different tasks complicate categorization by count, type, and memory, so calculated FLOPs may differ slightly from actual resources.Source: §4 · p. 9
• Conference computing statements remain insufficient reporting; the authors call for greater transparency and recognition of GPU availability, storage, and human labor.They frame these disclosures as integral to evaluating AI research and its long-term sustainability.Source: §4 · p. 9
• GPU and authorship measures omit failed-experiment costs and infrastructure costs, which vary across countries with GDP and AI policies.Source: §4 · p. 9

Conclusion

• Greater GPU power is associated with more advanced pretrained models, higher performance, and a greater likelihood of top-venue publication, concentrating influential AI research.The authors attribute performance advantages often to longer training on larger models and datasets.Source: Conclusion · p. 9
• The authors say future disparities in access to vast compute may persist or widen, and point to efficient methods and shared resources as counterbalances.They also emphasize fair allocation and attention to environmental footprints.Source: Conclusion · p. 9

A Methodology

• FM research typically appears in peer-reviewed computer science conferences or journals, with some conferences requiring experimental-work checklists on computing resources and human subjects.Source: A Methodology · p. 14
• After submission, an area chair invites two to three expert reviewers, then decides acceptance or rejection based on their scores and feedback unless authors withdraw.Source: A Methodology · p. 14
• Authors may respond through rebuttal or revise-and-resubmit cycles intended to improve clarity and quality; conference acceptance rates generally range from 25% to 35%.Source: A Methodology · p. 14

A.1 Public Data Collection

• The study gathered papers from eight conferences, collecting 5,889 of 34,828 papers and adding 751 rejected or withdrawn ICLR papers; after title-based deduplication of 123 resubmissions, 6,517 unique papers remained.GPT-4o mini analyzed titles and abstracts five times, assigning domain, phase, and methodology by majority vote; where multiple categories applied, only the first mentioned was selected.Source: §A.1 · pp. 14–15

GPT-4o Prompt for Extracting Information from PDFs

• The PDF extraction prompt requests structured information on paper characteristics, computing use, models, funding, dataset and API costs, and other cost sources.Source: GPT-4o Prompt for Extracting… · pp. 16–17

B Survey Questions

• Survey questions drew on guidelines from eight conferences; ARR conferences used more detailed computing-resource checklists, and 56 ineligible or invalid responses were excluded.After cleaning, the researchers cross-referenced paper records; survey submissions also included work from 15 conferences and journals.Source: B Survey Questions · p. 17

Part I: Paper Information

• Part I collects paper identity and publication information, including title, paper link, venue, and publication status.Source: Part I: Paper Information · p. 17

Part II: Human Resource Cost Information

• Part II records project dates, contributor counts, submission and acceptance attempts, and funding region and type.Source: Part II: Human Resource Cost Information · p. 17

Part III: Computing Cost Information

• Part III asks whether computing resources were used and records GPU count, type, memory, and estimated single-GPU running time.Source: Part III: Computing Cost Information · p. 18

Part IV: Dataset Cost Information

• Part IV asks about the dataset platform, dataset curation cost, API cost, and data storage.Source: Part IV: Dataset Cost Information · p. 18

Part V: Other Information

• Part V invites respondents to report other costs and provide survey feedback; the appendix also includes a conference computing-resource checklist table.Source: Part V: Other Information · p. 18, Table 4 · p. 19