Source-linked AI summary
Comparing Bibliometric Statistics Obtained from the Web of Science and Scopus
Eric Archambault, David Campbell, Yves Gingras, Vincent Lariviere
TL;DR
The paper asks whether country-level bibliometric statistics are comparable across the Web of Science and Scopus, an important issue for large-scale comparisons. It compares macro-level outputs and impacts across the databases and finds extremely high agreement in country measures, rankings, and field-level results.
Problem
The paper addresses whether bibliometric statistics for comparing countries are comparable and stable across the Web of Science and Scopus.
Method
The study compares macro-level country paper and citation indicators from WoS and Scopus across 1996–2007, using harmonized records and common field categories.
Results
Country paper and citation counts and rankings correlate at R2 > .99, with very high correlations also when papers are broken down by field.
Takeaways & Limitations
WoS and Scopus offer robust, largely database-independent tools for measuring scientific production and citations at the country level.
Takeaways & Limitations
The comparison covers 1996–2007 because Scopus does not include cited references before 1996.
Abstract
from arXiv · showhide
For more than 40 years, the Institute for Scientific Information (ISI, now part of Thomson Reuters) produced the only available bibliographic databases from which bibliometricians could compile large-scale bibliometric indicators. ISI's citation indexes, now regrouped under the Web of Science (WoS), were the major sources of bibliometric data until 2004, when Scopus was launched by the publisher Reed Elsevier. For those who perform bibliometric analyses and comparisons of countries or institutions, the existence of these two major databases raises the important question of the comparability and stability of statistics obtained from different data sources. This paper uses macro-level bibliometric indicators to compare results obtained from the WoS and Scopus. It shows that the correlations between the measures obtained with both databases for the number of papers and the number of citations received by countries, as well as for their ranks, are extremely high (R2 > .99). There is also a very high correlation when countries' papers are broken down by field. The paper thus provides evidence that indicators of scientific production and citations at the country level are stable and largely independent of the database.
Background and research question
Because web versions limited large-scale bibliometric indicator production, earlier comparisons often used small samples. This paper instead compares macro-level country indicators and rankings from Scopus and Web of Science, including scientific fields.
- Earlier database comparisons often used small samples because web versions were limited for producing bibliometric indicators.These studies examined small samples of papers, researchers, citation rates, or h-indexes.
- The paper uses bibliometric production platforms to compare macro-level indicators from Scopus and Web of Science.The platforms were implemented on Microsoft SQL Server.
- The analysis compares country rankings by paper and citation counts for science overall and for natural-science and engineering fields.It examines both ranking correlations and country-level numbers in the two databases.
- Google Scholar is excluded because the paper considers it unsuitable for compiling macro-level bibliometric data.The comparison therefore focuses only on Scopus and the Web of Science.
Methods
The study compares WoS and Scopus records over 1996–2007 after harmonizing database data and applying a common field classification. Because document types are classified asymmetrically, all document types were retained for counting papers and citations.
- Methods: 1996–2007 data from WoS and Scopus were compared, with the period restricted because Scopus lacks cited references before 1996.The authors state that twelve years is generally sufficient for performance measurement.
- Methods: Records were transformed into SQL Server relational databases, country names were harmonized, and NSF journal-based categories defined natural-science and engineering fields.Journals outside the NSF classification were manually classified.
- Methods: Because WoS and Scopus classify and cover documents differently, the classical article, research-note, and review-article subset could not be reproduced comparably.Differences were especially substantial for editorials, letters, and reviews.
- Methods: All document types were retained when calculating papers and citations, most of which were journal articles.This choice avoided imposing a non-comparable document-type selection across the two databases.
Results
Across country-level output, citations, ranks, scientific fields, and nanotechnology, WoS and Scopus produce highly similar bibliometric results, with only limited ranking differences.
- R2 values are 0.995 for country paper counts and 0.991 for country rankings across WoS and Scopus.The top 25 countries are identical, and ranking changes never exceed two places.
- 50% of countries retain the same rank, while 85% change by no more than 5% and 95% by no more than 10%.
- R2 values remain consistently above 0.99 when country paper counts are compared across three-year periods.Country ranks are also highly correlated, indicating similarity even with smaller datasets.
- R2 values exceed 0.99 for country citation counts and citation-based ranks, and the top 25 countries are the same in both databases.Ranking differences among the top countries never exceed two places.
- Across eight natural-science and engineering fields, paper-count correlations exceed 0.99 except in clinical medicine, where R2 is 0.987.Correlations remain well above 0.99 even in lower-output fields such as mathematics, earth and space, and biology.
- In nanotechnology, R2 is 0.991 for paper counts and 0.967 for citations; rank correlations are 0.990 and 0.974, respectively.For countries with at least 100 papers, ranking differences never exceed six places for papers or seven for citations.
Conclusion
The Web of Science and Scopus provide highly consistent country-level measures of scientific output and citation impact, including at specialty level. The paper therefore supports their use as robust tools for country-level science measurement while identifying lower-scale comparisons for further testing.
- Country-level papers and citations are extremely correlated across Web of Science and Scopus, even for specialty-level subsets such as nanotechnology.
- The databases offer robust tools for measuring science at the country level despite differences in scope, data volume, and coverage policies.
- Further research should test whether the results hold at the institutional level and in fields such as the social sciences and humanities.