Source-linked AI summary
Measuring contextual citation impact of scientific journals
Henk F. Moed
TL;DR
The paper addresses the problem of comparing journal citation impact across fields with different citation practices and citation potentials. It develops SNIP, a source-normalized indicator based on citation relationships and field characteristics, and shows that it changes journal comparisons in ways that account for within-category differences. The paper also emphasizes that no single journal indicator is universally adequate.
Problem
Citation counts and journal impact measures are difficult to compare fairly because citation potential and citation characteristics vary across disciplines and specialties.
Method
SNIP measures contextual journal citation impact using a subject field defined as the papers citing the journal, while accounting for citation frequency, impact maturation, and database coverage.
Results
SNIP produces substantially different normalized comparisons from raw impact, including nearly equal values for some journals despite differing citation potentials and reversed rankings in another pair.
Takeaways & Limitations
SNIP supports context-sensitive comparison by correcting for differences in referencing practices, citation maturation, and database coverage between subject fields.
Takeaways & Limitations
No single indicator can fully express journal performance because scholarly communication is complex and journal performance is multidimensional.
Abstract
from arXiv · showhide
This paper explores a new indicator of journal citation impact, denoted as source normalized impact per paper (SNIP). It measures a journal's contextual citation impact, taking into account characteristics of its properly defined subject field, especially the frequency at which authors cite other papers in their reference lists, the rapidity of maturing of citation impact, and the extent to which a database used for the assessment covers the field's literature. It further develops Eugene Garfield's notions of a field's 'citation potential' defined as the average length of references lists in a field and determining the probability of being cited, and the need in fair performance assessments to correct for differences between subject fields. A journal's subject field is defined as the set of papers citing that journal. SNIP is defined as the ratio of the journal's citation count per paper and the citation potential in its subject field. It aims to allow direct comparison of sources in different subject fields. Citation potential is shown to vary not only between journal subject categories - groupings of journals sharing a research field - or disciplines (e.g., journals in mathematics, engineering and social sciences tend to have lower values than titles in life sciences), but also between journals within the same subject category. For instance, basic journals tend to show higher citation potentials than applied or clinical journals, and journals covering emerging topics higher than periodicals in classical subjects or more general journals. SNIP corrects for such differences. Its strengths and limitations are critically discussed, and suggestions are made for further research. All empirical results are derived from Elsevier's Scopus.
1. Introduction
The paper motivates field-sensitive journal citation indicators because citation potential and citation practices vary across disciplines and specialties. It therefore develops SNIP as a source-normalized measure of contextual citation impact.
- Motivation: Citation counts should not be compared directly across research fields because citation potential varies substantially between them.Garfield relates citation potential to the average number of references per paper, noting 30 cited references in biochemistry versus 15 in mathematics.
- Motivation: Citation characteristics also differ in how quickly papers are cited, when citation rates peak, and how long citations continue.
- Motivation: Field distinctions may be too coarse because citation potential differs between specialties as well as disciplines, requiring compensation for these disparities.
- Related approaches: Prior relative indicators normalize a journal’s citation impact per paper against the world citation average in its subject field.
- Related approaches: Source normalization instead corrects for field differences in the frequency with which papers cite other documents, or the field’s citation potential.
- Contribution: The paper presents SNIP, which measures contextual citation impact using subject-field citation frequency, impact-maturation speed, and database coverage.The empirical results are derived from Elsevier’s Scopus.
2. Base principles
The paper defines SNIP by normalizing journal citation impact against the citation potential of a journal-specific subject field, while accounting for database coverage and citation maturation.
- 2. Base principles: SNIP evaluates journal citation impact using papers, citations, reference-list behavior, citation maturation, and database coverage.The study counts articles, conference proceedings papers, and reviews as peer-reviewed research papers in Scopus.
- 2. Base principles: A subject field is the collection of papers citing a journal, and their complete reference lists determine the field’s citation potential.Citation potential is the average number of cited references per source paper.
- 2. Base principles: Database citation potential counts cited references published in sources processed by Scopus rather than all cited references.About 80 per cent of cited references in 2007 Scopus source documents were published in Scopus-processed sources, with substantial disciplinary variation.
- 2. Base principles: The citation-impact numerator, raw impact per paper (RIP), counts citations in 2007 to papers published during 2004-2006.This three-year cited-paper window gives impact more time to mature than the Journal Citation Reports window.
- 2. Base principles: SNIP equals RIP when relative database citation potential is one; it exceeds RIP below one and falls below RIP above one.Relative database citation potential is normalized against the median journal’s database citation potential, which is 6.87 for Journal of Electronic Materials.
- 2. Base principles: 2.10 versus 1.12: RIP’s distribution standard deviation is almost twice SNIP’s, indicating a more concentrated SNIP distribution.The SNIP distribution is also less right-skewed, with lower highest scores than RIP.
3. Results
Table 6 shows that source normalization brings journals with substantially different raw impacts and citation potentials into comparable SNIP values, including differences within shared subject categories.
- SNIP values can be nearly identical despite substantial differences in relative database citation potential and raw impact per paper.The table is designed to show how source normalization changes rankings based on raw impact per paper.
- 1.81 versus 2.31: Journals of Gerontology A has 35% higher raw impact per paper but 22% lower SNIP than Journals of Gerontology B.Their relative database citation potentials are 2.02 and 1.17, respectively.
- 1.97 versus 1.98: Journal of Logic and Algebraic Programming and Journal of Differential Geometry have almost identical SNIP values despite RDCP values of 0.95 and 0.45.The journals are predominantly cited by computer science and purely mathematical sources, respectively.
- 1.67: Insect Biochemistry and Molecular Biology and Plant Molecular Biology have identical SNIP values despite raw impacts of 2.84 and 4.27.Their relative citation potentials are 1.70 and 2.55, while the latter journal’s raw impact is 50% higher.
- Emerging-topic journals tend to have higher citation potentials than journals covering classical topics or broad subject ranges.Examples include Journal of Nanoparticle Research versus Journal of Molecular Spectroscopy, and Nanotechnology versus Combustion Science & Technology.
- 1.21 against 1.24: Behaviour and Physiology & Behavior have different citation potentials and raw impacts, but almost equal SNIP values.The journals’ citation potentials are 1.47 and 2.36, and their raw impacts per paper are 1.78 and 2.93.
4. Discussion and conclusions
SNIP is presented as a contextual journal-impact indicator whose interpretation depends on how fields are delimited, citation practices vary, and databases cover the literature. The authors identify several strengths, limitations, and priorities for further research.
- SNIP defines a journal’s subject field through citation relationships rather than predefined journal categories, using a paper-by-paper delimitation.
- SNIP corrects for differences in referencing practices, citation-impact maturation, and database coverage across subject fields.Lower database coverage produces lower database citation potentials, which tends to yield higher SNIP values than raw impact per paper.
- SNIP can distinguish citation-context differences between journals covering distinct topics, approaches, or research objects within the same subject category.
- SNIP remains sensitive to journal composition because it does not correct for the fraction of review articles, and document-type categorization affects SNIP as it affects RIP.The percentage of reviews and a breakdown of papers by document type should provide background when interpreting SNIP.
- Journal impact indicators should not be used as surrogates for the citation impact of individual papers or research-group publication oeuvres.The authors state this limitation applies to both raw and source-normalized impact per paper.
- Further research could refine field delimitation, examine bias from papers that cite a journal outside the applied time window, and compare rankings with peer judgments.
APPENDIX A1: Mathematical model and methodological details
The appendix formalizes SNIP by defining journal subject fields, citation potential, database coverage, and citation-impact normalization, then tests sensitivity to time-window choices.
- Subject field delimitation: A journal’s subject field is the set of papers citing it in a particular year, typically papers citing content from the preceding ten years.The citation window used to delimit the field differs from the three-year window used to calculate citation-impact indicators.
- Citation potential and database coverage: Citation potential R_j is the arithmetic mean number of cited references in papers belonging to journal j’s subject field.The database citation potential restricts counted references to sources processed in the database.
- Citation potential and database coverage: f_j measures the fraction of cited references published in database-processed sources and indicates the internal database coverage of a journal’s subject field.This adjustment accounts for Scopus processing sources beyond scientific journals, although the paper labels all processed sources as journals.
- Raw and source normalized impact per paper: SNIP is defined as a normalized citation-per-paper ratio expressed per unit of database citation potential in the journal’s subject field.The appendix first defines raw citation impact per paper as citations to recent papers divided by the number of those papers, then normalizes it.
- Effects of variations in publication and citation time windows: Changing the fixed citing year has the largest effect on RIP and SNIP, while shortening the subject-field delimitation window has the smallest effect.For half of journals, changing the citing year changes SNIP by at most 11.7%, whereas the latter variation changes SNIP by at most 7.6%.
- Effects of variations in publication and citation time windows: SNIP varies only slightly more than RIP across the tested time-window configurations, with somewhat larger variation among all journals than among bigger journals.Bigger journals publish at least 100 papers per year during 2004–2006.