Source-linked AI summary
Characterizing in-text citations in scientific articles: A large-scale analysis
Kevin W. Boyack, Nees Jan van Eck, Giovanni Colavizza, Ludo Waltman
TL;DR
Full-text citation locations and contexts could support richer citation analytics, but a detailed understanding of citation characteristics is needed first. This study analyzes in-text citations across two large full-text databases and finds field-level differences in citation distributions, alongside temporal changes in citation activity.
Problem
A detailed understanding of in-text citation characteristics is needed before pursuing advanced full-text applications such as citation classification and impact measures.
Source: §1 · p. 1Method
Results
Field-level differences appear in citation position, reference age, and citation counts; sentence, reference, and mention counts increased over time across fields.
Source: §5 · p. 20Takeaways & Limitations
The quantified high-level distributions provide a basis for further studies of full-text features using syntactic and semantic analyses.
Source: §5 · p. 21The study covers only a relatively modest share of recent articles, considers five high-level fields, and does not normalize documents to IMRaD structure.
Source: §5 · p. 20
Abstract
from arXiv · showhide
We report characteristics of in-text citations in over five million full text articles from two large databases - the PubMed Central Open Access subset and Elsevier journals - as functions of time, textual progression, and scientific field. The purpose of this study is to understand the characteristics of in-text citations in a detailed way prior to pursuing other studies focused on answering more substantive research questions. As such, we have analyzed in-text citations in several ways and report many findings here. Perhaps most significantly, we find that there are large field-level differences that are reflected in position within the text, citation interval (or reference age), and citation counts of references. In general, the fields of Biomedical and Health Sciences, Life and Earth Sciences, and Physical Sciences and Engineering have similar reference distributions, although they vary in their specifics. The two remaining fields, Mathematics and Computer Science and Social Science and Humanities, have different reference distributions from the other three fields and between themselves. We also show that in all fields the numbers of sentences, references, and in-text mentions per article have increased over time, and that there are field-level and temporal differences in the numbers of in-text mentions per reference. A final finding is that references mentioned only once tend to be much more highly cited than those mentioned multiple times.
1 Introduction
Full-text articles provide citation locations and contexts that can support richer citation analytics, but the paper first establishes the characteristics of in-text citations. It examines their distributions across two large databases, with attention to field differences and textual progression.
1 Introduction
- Full text exposes citation locations and contexts that could support citation classification, impact measures weighted by citation features, and document clustering.These applications are beyond the scope of this paper.Source: §1 · p. 1
- A reference is a bibliography item, while an in-text citation or mention is each occurrence of that reference within the document.A reference may be mentioned once or multiple times.Source: §1 · p. 2
- The study examines in-text citation distributions in the PubMed Central Open Access subset and a large portion of the Elsevier corpus.It investigates field-level dependencies and citation counts as a function of textual progression.Source: §1 · p. 2
2 Background
Prior work has examined citation position, frequency, and value using both small hand-coded samples and larger full-text collections. Studies report links between citation location or repeated mentions and reference importance, while earlier distribution studies found recurring concentration patterns.
2 Background
- Early small-sample studies suggested that citation location, mention count, time, and discipline may relate to citation value or patterns.Voos & Dagaev (1976) reported disciplinary differences, while Bonzi (1982) found mention counts promising for predicting relatedness.Source: §2 · p. 2
- Several studies associated more frequent mentions or later-section citations with greater reference importance or utility.Evidence includes Hooten's comparison of multiply and singly cited references, author assessments by Tang & Safer (2008), and citation-function studies by Cano (1989) and Maričić et al. (1998).Source: §2 · pp. 2–3
- Recent work tested mention counts, citation density, and section-specific mentions as predictors of citation strength or influence.Studies also proposed filtering some background and introduction citations to focus on influential references.Source: §2 · pp. 3–4
- Bertin et al. (2016) found similar mention and reference-age distributions across PLOS journals after organizing articles by IMRaD structure.They reported mentions concentrated in introductions and discussions, while reference ages varied across article sections.Source: §2 · p. 4
- Earlier distribution studies generally found mentions concentrated toward article beginnings and endings, with Hu et al. (2013) reporting half in the first 30% of text.Hu et al. (2017) also examined multiply mentioned references, while Hu et al. (2013) and Ding et al. (2013) counted mentions by section.Source: §2 · p. 4
3 Data and Methods
The study analyzes machine-readable full text from PMCOA and Elsevier, processing each source separately and examining citation distributions across five broad scientific fields. The datasets differ in coverage and processing, while several shared descriptive measures are similar.
3 Data and Methods
- The analysis draws on PMCOA and Elsevier full text, sources with broad but distinct publication coverage.PMCOA includes roughly 36% of PubMed Central articles; Elsevier data cover almost 3,000 journals across scientific and technical fields.Source: §3 · p. 5
3.1 PubMed Central Open Access Subset
- PMCOA records were filtered by article type, publication year, and reference-year validity before text and citation positions were parsed.The analysis included journal articles and reviews published from 1998 onward with at least one reference meeting the stated year criteria.Source: §3.1 · p. 5
3.2 Elsevier Full Text
- Elsevier records were filtered to accessible English full-text articles, short communications, and reviews, then parsed and matched to Web of Science.Only major sections were considered, and reference citation counts were determined for works indexed in the Web of Science database used by CWTS.Source: §3.2 · pp. 6–7
3.3 Descriptive Statistics
- The two datasets had similar average mentions per reference and sentence-level mention rates, despite differences in article length and section-counting practice.Both averaged 1.57 mentions per reference; 19.9% of PMCOA sentences and 19.8% of Elsevier sentences contained mentions.Source: §3.3 · p. 7, Table 1 · p. 7
- The Elsevier analysis linked 3,892,083 publications from 2000–2015 to five Leiden Ranking fields, with BHS and PSE largest and MCS and SSH smallest.BHS and PSE each represented over 35% of content, while MCS and SSH each represented around 6%.Source: §3.3 · p. 7, Table 2 · p. 8
4 Results
Citation patterns vary by field in their position within articles, reference ages, and citation counts. Across the datasets, sentence and citation measures also change over time, while mention frequency relates differently to citation counts across fields.
4.1 Temporal Trends
- Sentence, reference, and mention measures generally increased over time, with field differences in their levels and growth rates.Sentence counts rose about 35% in MCS and PSE from 2000 to 2015, compared with 20% in BHS and LES; the percentage of sentences containing mentions rose most in SSH, by 39%.Source: §4.1 · pp. 10–11, Figure 2 · p. 10, Figure 3 · p. 12
4.2 Distributions by Mentions
- In 2015, 71.46% of PMCOA references and 69.48% of ELS references were mentioned only once.Across fields, citation intervals generally decreased as mention counts rose, while SSH references were oldest and BHS references youngest.Source: §4.2 · pp. 13–14, Table 4 · p. 14, Figure 5 · p. 15
- References mentioned once tend to have higher citation counts than those mentioned multiple times, except that MCS and SSH show more gradual decreases.In BHS, LES, and PSE, references mentioned five or more times have less than half the citation counts of references mentioned once; citation counts are not normalized for reference age.Source: §4.2 · pp. 14–15, Figure 5 · p. 15
4.3 Distributions by Text Progression
- Mention distributions across text progression differ by field, and their general placement pattern changed little between 2005 and 2015.MCS and PSE lack the secondary peak seen in aggregate datasets, while SSH declines more gradually; the authors note aggregate measures and avoid assigning mentions to article sections.Source: §4.3 · pp. 16–17, Figure 6 · p. 17
- BHS, LES, and PSE citation profiles peak at different text positions, while MCS and SSH show distinct profiles of their own.For BHS, LES, and PSE, peaks occur around the traditional methods location; MCS peaks at the 55th centile, while SSH peaks at the 60–65th centile.Source: §4.3 · pp. 18–19, Figure 8 · p. 19
- Citation counts across text progression also vary by field, with SSH and MCS profiles differing from those of BHS, LES, and PSE.Citation counts were not age-normalized, and references not identifiable in WoS or Scopus were excluded.Source: §4.3 · pp. 18–19, Figure 7 · p. 18, Figure 8 · p. 19
5 Conclusions
Across five million articles, citation distributions differ by field in textual position, reference age, and citation counts. The conclusions also identify shared temporal increases and limits on how broadly the field-level patterns can be generalized.
5 Conclusions
- BHS, LES, and PSE have broadly similar reference distributions, while MCS and SSH differ from those fields and from each other in position and citation-count peaks.MCS and SSH have more references in the 15th–50th centile range and citation-count peaks near the 60th rather than the 30th centile; the authors say semantic as well as syntactic analysis is needed to explore why.Source: §5 · p. 20
- Sentence, reference, and mention counts increased over time in every field, while mentions per reference stayed nearly constant corpus-wide but rose in MCS and SSH and fell in PSE.References mentioned once were much more highly cited than those mentioned many times.Source: §5 · p. 20
- The study covers a modest share of recent articles and only five broad fields, so additional data could reveal other patterns and within-field differences may be substantial.The authors also did not normalize documents to IMRaD section structure, leaving structural variation unaccounted for.Source: §5 · p. 20
- Despite those limitations, the authors characterize the study as robust and report that it covers 100 times more articles and references than Bertin et al. (2016).They propose further syntactic and semantic analyses of full-text articles, with potential relevance to citation theory, information search and retrieval, and modeling science.Source: §5 · p. 21