Source-linked AI summary
Google Scholar, Web of Science, and Scopus: a systematic comparison of citations in 252 subject categories
Alberto Martín-Martín, Enrique Orduna-Malea, Mike Thelwall, Emilio Delgado López-Cózar
TL;DR
Prior studies provided limited and inconsistent evidence about how Google Scholar, Web of Science, and Scopus differ in citation coverage, despite their use in research evaluation. This study systematically compares the three databases across 2,299 highly cited documents from 252 subject categories, finding that Google Scholar identifies substantially more citations while largely preserving document-level citation rankings.
Problem
Prior studies examined limited and inconsistent sets of articles, leaving a need for comprehensive evidence about citation-source differences across Google Scholar, Web of Science, and Scopus.
Method
The study compares citations from Google Scholar, the Web of Science Core Collection, and Scopus for highly cited English-language documents published in 2006 across 252 subject categories.
Results
Google Scholar found significantly more citations than Web of Science and Scopus across all subject areas, while finding 95% of Web of Science citations and 92% of Scopus citations.
Takeaways & Limitations
Google Scholar provides broader citation coverage, especially where Web of Science and Scopus coverage is poorer, but its additional citations are often lower-impact non-journal or non-English sources.
Takeaways & Limitations
Because the sample was selected from Google Scholar, the comparisons may give Google Scholar an advantage, although the authors report that citation-count correlations suggest this advantage is not substantial.
Abstract
from arXiv · showhide
Despite citation counts from Google Scholar (GS), Web of Science (WoS), and Scopus being widely consulted by researchers and sometimes used in research evaluations, there is no recent or systematic evidence about the differences between them. In response, this paper investigates 2,448,055 citations to 2,299 English-language highly-cited documents from 252 GS subject categories published in 2006, comparing GS, the WoS Core Collection, and Scopus. GS consistently found the largest percentage of citations across all areas (93%-96%), far ahead of Scopus (35%-77%) and WoS (27%-73%). GS found nearly all the WoS (95%) and Scopus (92%) citations. Most citations found only by GS were from non-journal sources (48%-65%), including theses, books, conference papers, and unpublished materials. Many were non-English (19%-38%), and they tended to be much less cited than citing sources that were also in Scopus or WoS. Despite the many unique GS citing sources, Spearman correlations between citation counts in GS and WoS or Scopus are high (0.78-0.99). They are lower in the Humanities, and lower between GS and WoS than between GS and Scopus. The results suggest that in all areas GS citation data is essentially a superset of WoS and Scopus, with substantial extra coverage.
1. Introduction
Google Scholar’s broad web-based indexing offered potentially more comprehensive scholarly coverage, but systematic comparisons of its citation coverage with Web of Science and Scopus remained limited and inconsistent.
- Google Scholar indexed documents beyond selected source lists, potentially expanding coverage of scholarly literature.
- Google Scholar citation data became usable for research evaluations because it was freely available and could be gathered with Publish or Perish.
- Google Scholar has known indexing errors but showed strong disciplinary and language coverage, especially in Humanities and Social Sciences.
- Prior citation-source studies examined small or inconsistent samples, with Google Scholar’s unique-citation share ranging from 13% to 67%.The only multidisciplinary study covered articles from 12 journals across 6 subject areas.
4 Supplementary materials available from https://dx.doi.org/10.31235/osf.io/pqr53
The study identifies a need for comprehensive, updated comparisons of citation coverage across Google Scholar, Web of Science, and Scopus to inform citation-based evaluations.
- Earlier studies compared unique and overlapping citations across Google Scholar, Web of Science, and Scopus using varied samples and disciplinary coverage.
- Prior research covered fields including Library and Information Science, Medicine, Physics, Chemistry, Computer Science, Biology, and several social-science areas.
- A comprehensive cross-subject analysis was needed because previous evidence about citing-source coverage was limited and required updating.
- The investigation addressed overlap, document-type differences in unique Google Scholar citations, and similarity between citation counts across databases.
2. Methods
The study compared citations to highly cited documents across 252 Google Scholar subject categories using records extracted from Google Scholar, Web of Science, and Scopus.
- The sample comprised 2,515 English-language highly cited documents published in 2006 and classified into 252 Google Scholar subject categories.The analysed set was not a random sample of academic publications.
- The researchers extracted citations from all three databases and analysed citations to the 2,299 documents covered by each database.Google Scholar supplied 2,415,072 citations overall, reduced to 2,301,997 for the shared document set.
- Google Scholar records were cleaned and enriched with DOI and metadata information from publisher APIs, CrossRef, DataCite, and HTML metadata.
- Document types and languages were classified using combined database, API, webpage, and manual-sample information.Language was identified for 98% of citing documents, while metadata sources covered document type for 85%.
- Citation records were matched pairwise across databases using DOI matches followed by title-similarity and author checks.
- For document-level comparisons, Spearman correlations were calculated by subject category between Google Scholar and Web of Science or Scopus citation counts.
3. Results
GS found the largest share of citations across subject areas, with especially extensive coverage beyond WoS and Scopus in the humanities and social sciences. Its unique citations were more diverse in document type and language, while citation-count correlations with other databases remained high.
- Citation overlap: 36.9% of all citations were found only by GS, whereas 46.9% were found by all three databases.Only 6% were found by WoS and/or Scopus but not by GS.
- Citation coverage: 94% of all citations were found by GS, compared with 52% by WoS and 60% by Scopus.GS found 95% of WoS citations and 92% of Scopus citations.
- Document types: Unique GS citations exceeded 50% in all broad subject areas except Health & Medical Sciences, where non-journal sources comprised 48%.Theses or dissertations were the most frequent non-journal type, while books, conference papers, and unpublished papers varied by field.
- Document types: Unknown document types ranged from approximately 20% in Special Education and Ethnic & Cultural Studies to over 50% in Quantum Mechanics, Geometry, and Algebra.These category-level estimates lacked the correction factor applied in the broad-subject analysis.
- Languages: Unique GS citations were 62%-80% English across broad subject areas, compared with 97%-100% English among non-unique citations.Chinese accounted for 4%-12% of unique GS citations, and another 5%-10% used languages outside the 11 most frequent overall.
- Citation counts: Spearman correlations between GS and WoS ranged from .78 in Literature to .98 in several science categories.GS found more citations on average than WoS and Scopus, with stronger correlations associated with lower mean citation ratios.
4. Discussion
The discussion compares the findings with earlier studies and identifies methodological and scope limitations. It emphasizes broader Google Scholar coverage while cautioning that matching procedures and database access affect the reported overlaps.
- Limitations: Selecting highly cited documents through GS may advantage GS, although high correlations suggest that this advantage is not substantial for document-level rankings.The study’s sample may also be less representative of documents citing scientific literature generally, especially for emerging areas and larger specialisms.
- Limitations: The reported overlaps do not apply to all possible WoS data because the study lacked access to Clarivate Analytics’ ESCI Backfile for 2005-2014 documents.Some citations classified as unique to GS and/or Scopus may also be captured by ESCI.
- Limitations: Conservative citation matching likely produced false negatives, particularly where DOI use was limited, potentially understating overlaps and overstating unique-citation percentages.The matching relied more often on strict title-similarity rules in such categories.
- Comparison with previous studies: GS coverage has become more comprehensive over time, with current coverage higher and more consistent across subject areas than in earlier studies.GS found 94.3% of citations in Chemical & Material Sciences, compared with 38% of Chemistry citations reported by an earlier study.
- Comparison with previous studies: GS found 88%-97% of WoS citations and 84%-94% of Scopus citations in the current study, exceeding most previously reported overlap ranges.The current study also reports lower percentages of unique WoS and Scopus citations than previous research.
- Citation correlations: Citation-count correlations are strong between GS and WoS or Scopus, ranging from .78 to .98 for GS-WoS and .92 to .99 for GS-Scopus.The GS-WoS correlation is weakest in Literature, while GS-Scopus correlations are generally stronger.
5. Conclusions
The study finds that Google Scholar provides broader citation coverage than the WoS Core Collection and Scopus across subject areas, but its extra citations often come from less conventional sources. It concludes that GS is comprehensive yet remains limited for large-scale evaluation by weak metadata and extraction constraints.
- Coverage: GS found significantly more citations than the WoS Core Collection and Scopus across all subject areas, with unique citations exceeding 50% in three broad areas.The areas are Humanities, Literature & Arts, Social Sciences, and Business, Economics & Management.
- Unique citations: 48%-65% of GS unique citations are non-journal sources, including theses, books, conference proceedings, unpublished materials, and other document types.These citations are primarily in English, but 19%-38% are in other languages and their scientific impact is lower on average than overlapping citations.
- Citation rankings: GS-WoS and GS-Scopus citation-count correlations are very strong across subject categories, although they are weaker in Humanities and for GS-WoS than GS-Scopus.The reported ranges are .78 to .98 for GS-WoS and .92 to .99 for GS-Scopus.
- Implications: GS citation counts may be useful when documents outside WoS or Scopus matter for an evaluation, but higher counts should not automatically be treated as superior measures of scholarly impact.Some additional citations may reflect educational impact, such as citations from Master’s theses.
- Practical limitations: GS has high citation-data comprehensiveness, but limited metadata and no reliable scalable extraction method reduce its practical suitability for large-scale citation analyses.Manual data collection remains possible for small-scale uses, and combining GS data with rich open data could address metadata limitations.