Source-linked AI summary
The Journal Coverage of Web of Science and Scopus: a Comparative Analysis
Philippe Mongeon, Adele Paul-Hus
TL;DR
Because research evaluation relies heavily on bibliometric databases, this study compares Web of Science and Scopus journal coverage using Ulrich’s directory. It finds that both databases favor some fields and English-language journals, while differing substantially in coverage.
Problem
The study addresses limited evidence about how citation databases’ field and language coverage affects the validity of bibliometric research evaluation.
Method
The authors compare Web of Science and Scopus journal coverage across fields, countries, and languages using Ulrich’s academic periodical directory.
Results
Both databases favor Natural Sciences, Engineering, Biomedical Research, and English-language journals, while their country and language coverage remains unbalanced.
Takeaways & Limitations
Bibliometric evaluation results may vary by database, so database coverage characteristics matter when interpreting scientific output and impact.
Takeaways & Limitations
Ulrich’s data contain inconsistencies, including journals listed with multiple languages, raising questions about language classifications.
Abstract
from arXiv · showhide
Bibliometric methods are used in multiple fields for a variety of purposes, namely for research evaluation. Most bibliometric analyses have in common their data sources: Thomson Reuters' Web of Science (WoS) and Elsevier's Scopus. This research compares the journal coverage of both databases in terms of fields, countries and languages, using Ulrich's extensive periodical directory as a base for comparison. Results indicate that the use of either WoS or Scopus for research evaluation may introduces biases that favor Natural Sciences and Engineering as well as Biomedical Research to the detriment of Social Sciences and Arts and Humanities. Similarly, English-language journals are overrepresented to the detriment of other languages. While both databases share these biases, their coverage differs substantially. As a consequence, the results of bibliometric analyses may vary depending on the database used.
Limits of databases for the evaluation of research in social sciences and humanities
WoS and Scopus rely mainly on journals, making their assessments less representative where books and other non-journal outputs are important, especially in Social Sciences and Arts and Humanities. Their language coverage also creates biases, motivating an updated comparison of both databases across fields, countries, and languages against Ulrich’s directory.
- Publication formats: WoS and Scopus mainly cover journals rather than books, proceedings, and reports.This limits their representativeness of scientific activity when important knowledge is disseminated through other channels.
- Publication formats: Books are more frequent and more important for researchers’ careers in Arts and Humanities than articles.Articles dominate dissemination in Natural Sciences, Engineering, and Biomedical Research, but not in many Social Sciences disciplines or Arts and Humanities.
- Evaluation limits: WoS and Scopus therefore provide less accurate portraits of output and impact in Social Sciences and Arts and Humanities than in the natural and biomedical fields.The limitation applies at individual, institutional, and national levels of aggregation.
- Language coverage: Language biases in citation-database coverage complicate comparative evaluations of national research systems.WoS coverage has been observed to overrepresent English-language journals relative to Ulrich’s comprehensive worldwide periodical directory.
- Study purpose: The study revisits these issues by comparing WoS and Scopus coverage across fields, countries, and languages, using Ulrich’s extensive periodical directory as a benchmark.It examines whether previously identified WoS biases persist and whether similar biases occur in Scopus.
Methodology · Journal lists
The study used Ulrich’s Academic/Scholarly journal records as a comparison base, removing duplicate entries to produce a unique-journal set. It also compiled journal lists from Thomson Reuters’ Web of Science and Elsevier’s Scopus for comparison.
- Journal lists: 162,955 Ulrich’s records met the “Academic/Scholarly” criterion and were downloaded manually.The download process was lengthy because a maximum of 500 records could be downloaded at once.
- Journal lists: Ulrich’s database often listed one journal multiple times, including separate entries for different formats or languages.Examples included online versus print formats and publication in more than one language.
- Journal lists: 70,644 unique journals remained after duplicate entries were eliminated.
- Journal lists: The Thomson Reuters master journal list was downloaded from the Thomson Reuters website.
- Journal lists: Because Thomson Reuters did not provide a spreadsheet, source code was converted into a table using an XSL style sheet.
- Journal lists: 16,957 entries comprised Thomson Reuters’ combined list of SCI-Expanded, SSCI, and A&HCI journals.
- Journal lists: 34,274 titles comprised the Scopus journal list, which Elsevier provided directly in spreadsheet format.
- Journal lists: The Scopus list was last updated in May 2014.
Matching
WoS and Scopus journals were matched to Ulrich’s list using ISSNs, then titles with manual verification of remaining matches. After restricting the dataset to active journals, the final sample comprised 13,605 WoS and 20,346 Scopus matches among 63,013 active Ulrich journals.
- Matching procedure: Matching proceeded first by ISSN and then by journal title, with manual verification of title-based matches to eliminate false positives.
- Initial matches: 14,637 WoS journals and 23,189 Scopus journals were matched with Ulrich’s list before active-status filtering.
- Active-journal restriction: The analysis retained titles classified in Ulrich as “serial type = journal” and “status = active.”
- Final sample: 13,605 WoS and 20,346 Scopus matched journals remained in the final sample, drawn from 63,013 active Ulrich journals.
Journals classification · By field
Journal titles were classified by assigning every Ulrich’s subject to one of four broad fields in the National Science Foundation classification. The fields were Natural Sciences and Engineering, Biomedical Research, Social Sciences, and Arts and Humanities.
- By field: Every Ulrich’s subject was assigned to a broad discipline classification.This classification was applied to journal titles.
- By field: The classification used four broad fields from the NSF classification.The source specifies NSF (2006) as the classification framework.
- By field: Natural Sciences and Engineering formed one of the four broad fields.
- By field: Biomedical Research formed another broad field in the classification.It included the biomedical research and clinical medicine NSF categories.
- By field: Biomedical Research excluded health, which was classified as part of Social Sciences.
- By field: Arts and Humanities constituted the fourth broad field.
By country
Journals were assigned countries using the publisher’s country listed in Ulrich. Journals with multiple publisher countries were counted fully for each country rather than fractionally.
- By country: Country assignment used the publisher’s country provided by Ulrich, the database’s only available geographical information.Most journals had only one publisher’s country.
- By country: Journals with multiple publisher countries were assigned to all reported countries using full counting.This included cases where print and online versions were published in different countries.
- By country: A journal published in Canada and the United States counted as one journal for each country, rather than 0.5 for each under fractional counting.Full counting therefore differs from assigning fractional country shares.
By language … Relative distribution of journals by field, country and language
The study defined journal languages from Ulrich records, counted only the languages of full-text articles, and assessed database coverage against Ulrich across fields, countries, and languages. It used relative distributions to identify potential overrepresentation or underrepresentation in WoS and Scopus.
- By language: Journal languages were assigned from Ulrich records using the full counting method.The procedure used the language information recorded for each journal.
- By language: Only the language of the journal’s full-text articles was included in the analysis.Additional languages used for abstracts, comments, or notes were excluded.
- Databases coverage: Relative database coverage was calculated by dividing each database’s journal count by the corresponding number of journals in Ulrich.This calculation was performed separately for each of the four broad research fields.
- Relative distribution of journals by field, country and language: Database journal distributions were compared with Ulrich’s distributions to assess field overrepresentation or underrepresentation.The comparison evaluated whether database composition differed from the Ulrich reference distribution.
- Relative distribution of journals by field, country and language: Such distributional differences may cause cross-field comparisons of scientific output and impact to favor some fields over others.The paper treats overrepresentation or underrepresentation as an indicator of potential comparative bias.
- Relative distribution of journals by field, country and language: Relative distribution was computed by dividing each field’s proportion in WoS or Scopus by its proportion in Ulrich.The same approach was applied to countries and languages within each field.
Coverage overlap of Web of Science and Scopus
The study assessed how much journal coverage overlaps between Web of Science and Scopus by classifying journals into three categories: WoS only, Overlap, and Scopus only.
- Classification categories: Journals were classified into three categories to assess coverage overlap between WoS and Scopus.The categories distinguish database-specific coverage from shared coverage.
- Classification categories: WoS only comprises journals indexed in WoS but not in Scopus.
- Classification categories: Overlap comprises journals indexed in both WoS and Scopus.
- Classification categories: Scopus only comprises journals indexed in Scopus but not in WoS.
Results
WoS and Scopus differ substantially in journal coverage across fields, countries, and languages, despite sharing biases toward Natural Sciences and Engineering, Biomedical Research, and English-language journals. Scopus generally covers more journals and has more exclusive titles, while overlap varies by field and country.
- Field coverage: Scopus covers almost half of Biomedical Research journals versus about 28% for WoS, whereas coverage is closest in Natural Sciences and Engineering at 38% and 33%, respectively.Social Sciences and Arts and Humanities receive less coverage: Scopus indexes less than 25% of journals in each, while WoS indexes less than 15%.
- Coverage overlap: Scopus includes most WoS-indexed journals except in Natural Sciences and Engineering, has more exclusive journals in every field, and generally covers more journals overall.Natural Sciences and Engineering has the highest number of WoS-exclusive journals relative to the other fields.
- Country coverage: Publishing-country coverage rankings are broadly similar between WoS and Scopus, but Spanish and Indian journals have much larger relative coverage in Scopus than in WoS.The Netherlands, the United Kingdom, and the United States are overrepresented in every field, while most other countries are underrepresented.
- Language coverage: English is the only language consistently and strongly overrepresented in both databases and across all four fields, while most other languages are underrepresented.Natural Sciences and Engineering is an exception for WoS, where nine languages are overrepresented; language representation can diverge sharply between databases, as with Ukrainian and French in Biomedical Research.
Discussion
The study finds persistent country and language overrepresentation in both WoS and Scopus, despite Scopus’s larger coverage. Their differing journal overlap can make comparative analyses tool-dependent, while country and language definitions limit interpretation.
- Findings: Both WoS and Scopus overrepresent journals from certain countries and languages, disadvantaging others.Countries such as the United States, United Kingdom, Netherlands, France, Germany, and Switzerland occupy larger shares of indexed journals than of Ulrich’s journals.
- Findings: WoS and Scopus differ most in Natural Science and Engineering and Arts and Humanities, where overlap is lowest and WoS-exclusive coverage is highest.This divergence increases tool-dependency in comparative analyses using either database.
- Findings: 21% of Chinese Natural Science and Engineering journals are indexed by both databases, compared with 78% of Singaporean journals.Analyses of Chinese journals may therefore vary greatly by source, whereas Singaporean analyses may produce more coherent results.
- Limitations: The study’s main limitations arise from defining a journal’s country and language.Publisher country was used because of data availability, although it may convey primarily financial information.
- Limitations: Ulrich’s language data inconsistently distinguishes explicit language lists from “multiple languages,” but such cases represent only 1.4% of indexed journals.The ambiguity includes journals listing English, French, German, Spanish, and Dutch, yet its potential effect on results is considered minimal.
Further research
The study portrays WoS and Scopus journal coverage using field, publisher-country, and journal-language classifications from Ulrich’s periodical database. Further research should examine article-level language, local-journal representation, and journals’ national versus international identities.
- The study characterizes WoS and Scopus journal coverage by field, publisher country, and journal language, using information from Ulrich’s periodical database.
- Further research could analyze article-level language to better measure a journal’s proportional language coverage.
- Research could investigate whether national journals focused on local matters are adequately represented in WoS and Scopus or whether the databases favor an international focus.
- Further research could assess journals’ national versus international identities through subjects, editorial boards, and author nationalities.Science illustrates this issue: 90% of its published articles come from American authors despite its international aim.
Conclusion
WoS and Scopus both provide unbalanced journal coverage that favors Natural Sciences, Engineering, Biomedical Research, and English-language journals, while underrepresenting Social Sciences, Arts and Humanities, countries, and other languages. Research evaluation should therefore account for each database’s coverage limits and select tools suited to the task.
- Coverage biases: WoS underrepresents Social Sciences and Arts and Humanities relative to their share in Ulrich, while English-language overrepresentation remains persistent.The English-language bias identified previously was still confirmed by the study’s data.
- Coverage biases: Despite Scopus’s larger journal coverage in all fields, both databases favor Natural Sciences, Engineering, Biomedical Research, and English-language journals.These biases disadvantage Social Sciences, Arts and Humanities, and journals in languages other than English.
- Implications: These coverage biases should be considered when assessing scientific activities and drawing conclusions from bibliometric analyses or research evaluations.The study emphasizes understanding each tool’s strengths and limits before choosing it for a task.
- Alternative tools: Field-specific citation indexes can provide more complete portraits of covered fields and may be better suited for field-specific research evaluation.Examples include Chemical Abstracts Services for chemistry and CAB Abstracts for agriculture, environment, veterinary sciences, applied economics, food science, and nutrition.
- Alternative tools: National citation indexes can represent local scientists’ research more completely and assess national research impact more accurately, but they are not suitable alternatives for multidisciplinary and international analyses.The paper notes examples including the Indian, Serbian, and Thai-Journal Citation Indexes.