A Glimpse of the First Eight Months of the COVID-19 Literature on Microsoft Academic Graph: Themes, Citation Contexts, and Uncertainties

Introduction

• The COVID-19 literature grew rapidly across disciplines as the pandemic created urgent scientific, social, economic, and political challenges.CORD-19 began with over 29,000 articles and reached 130,000 by early September 2020.Source: Introduction · p. 1
• CORD-19 and similar datasets can lack cited references, abstracts, or adequate data coverage and quality, limiting citation-based network analyses.Citation-based methods such as co-citation analysis use references to study research, impact, and higher-level concepts.Source: Introduction · p. 2
• Microsoft Academic Graph provides flexible access to publication metadata, cited references, and citation contexts: citing sentences together with surrounding sentences.Microsoft Academic is also a major source behind CORD-19.Source: Introduction · p. 2
• The method supports building CORD-19-like datasets with cited references and citation contexts, and expands scientometric and visual analyses to finer-grained citation contexts and uncertainties.It also enables interactive exploration of uncertainties associated with citations in an integrated visual analytic environment based on CiteSpace.Source: Introduction · pp. 2–3

CiteSpace

• CiteSpace maps relationships among references into co-citation clusters, guiding analysis of thematic concentrations, bridges, representative articles, and cluster meaning.Its workflow examines cluster interrelationships from high-level aggregation to individual references.Source: CiteSpace · p. 3
• Cluster decomposition quality is assessed with network modularity, cluster silhouette scores, and a size-weighted silhouette mean, with values nearer 1 indicating greater clarity.Source: CiteSpace · p. 3
• CiteSpace uses high betweenness centrality to flag structural-hole nodes, citation-per-year tree rings to show highly cited articles, and citation bursts to identify sharp attention increases.Structural holes are associated with creativity, originality, and boundary-spanning.Source: CiteSpace · p. 3
• Structural Variation Analysis identifies newly published articles with transformative potential when they connect previously disjoint thematic areas, but such potential is not an actual change unless the community consolidates it.Source: CiteSpace · p. 3
• CiteSpace measures epistemic, hedging, and transitional uncertainty linguistically with information entropy, aggregating uncertainty across passages, articles, and reference clusters.Epistemic cues signal unknown or contested aspects; hedging words qualify certainty; transitional words can signal argument shifts or alternative interpretations. Microsoft Academic Services is identified as an exception among commonly used bibliographic databases for access to citation contexts.Source: CiteSpace · pp. 3–4

Microsoft Academic Services

• MAS enables retrieval of citing articles and their citation contexts, and the study integrates visual analysis of those contexts with citation-based analysis.The authors chose MAS over CORD-19 because that dataset lacked cited references; they describe their construction approach as flexible and extensible through CiteSpace.Source: Microsoft Academic Services · p. 4
• The September 5, 2020 dataset contains 79,476 records, including 7,693 with citation contexts, and captures the first eight months of 2020 COVID-19 literature on MAG.It was constructed from MAS using fields of study; 77,897 records had been cited at least once.Source: Microsoft Academic Services · p. 5
• At the first two analysis levels, the dataset supports CiteSpace co-citation visualizations and integrates citation contexts to measure three types of uncertainty that can help organize extensive narratives.Uncertainty scores highlight information potentially valuable to sense-making; a reference may appear in multiple contexts within one or more citing articles.Source: Microsoft Academic Services · p. 5
• At the third level, the study applies the SVA procedure to identify newly published articles that have low citation counts but show signs of transformative potential.Source: Microsoft Academic Services · p. 5

Data

• The MAG dataset included publications dating as early as 2014 and future-dated records, but analysis focused on the first eight months of 2020.The dataset was constructed by matching fields of study.Source: Data · p. 5
• CORD-19 contained 130,000 articles; Dimensions and Lens full-text searches each returned over 100,000, while Web of Science represented about 23% and MAG over 62% of CORD-19’s volume.The comparison was based on searches run on September 5, 2020; metadata-only searches returned fewer articles than full-text searches.Source: Data · pp. 6–7
• The demonstrative dataset used fields of study to reduce query-formation complexity; Cascading Citation Expansion was cited as a more sophisticated way to optimize collection quality.The comparison searches used identical or closely matched queries across bibliographic databases.Source: Data · p. 6

Overview

• Clusters identify varied themes, including wuhan china, renin angiotensin aldosterone system, spike protein, pregnant women, dental practice, chest [CT], and mental health.Cluster size and thematic concentration vary; users may adjust the network parameters.Source: Overview · p. 7
• Citation contexts extend conventional Document Co-Citation Analysis by retaining frequently cited references and discarding below-average ones, an option users can control.Conventional DCA typically weights each reference equally for a citing article because only its reference list is available.Source: Overview · p. 8
• Figure 1 maps co-citation clusters among 1,330 top-cited articles in a dataset of 77,897 COVID-19 articles, with clusters representing thematic concentrations.Examples include #0 wuhan china, #1 renin angiotensin aldosterone system, and #3 spike protein.Source: Overview · p. 7, Figure 1 · p. 8

Concept Trees

• In Cluster #5 on pregnant women, vertical transmission is the concept tree’s root and primary concern, with major branches highlighting other cluster themes.Figure 2 links the concept to citation contexts for interactive inspection.Source: Concept Trees · p. 8, Figure 2 · p. 9
• The dataset contains 111,360 citation contexts from 7,693 citing articles, covering 9.88% of 77,897 citing articles with at least one dataset citation.The authors considered the available contexts sufficient to assess the integrative approach’s feasibility.Source: Concept Trees · p. 8
• For Li, Q., 2020, the tree highlights incubation period and mean incubation period as major themes, with 5.2 days frequently appearing in inspected contexts.Collecting values for a specific variable across many publications can facilitate systematic reviews and meta-analyses.Source: Concept Trees · p. 9, Figure 3 · p. 9

Uncertainties

• Citation contexts are scored by uncertainty-cue presence and frequency; semantically equivalent cues are learned from hand-picked seeds against a global Dimensions database of over 200 million articles.Epistemic cues include “controversial,” “contradictory,” “inconsistent,” “unknown,” and “uncertain.” Combining them with conclusion-related rhetorical words can connect uncertainty to specific concepts, as illustrated in Table 3.Source: Uncertainties · p. 10, Table 3 · p. 10
• Citation contexts flag unresolved evidence: pre-pandemic studies said smoking and nicotine down-regulate ACE2 [3, 4], while pandemic studies reported up-regulation [5] [6] [7]; equal gender proportions among 80 patients contradicted earlier male-susceptibility conclusions.The examples show how conflicting findings can leave conclusions about nicotine or smoking effects on ACE2 unsettled, and how a study’s findings may contradict earlier conclusions about susceptibility.Source: Uncertainties · p. 11
• Figure 4 displays citation contexts for (Li, Guan et al. 2020), which had the dataset’s largest epistemic uncertainty score, highlighting cues such as “unknown,” “uncertainty,” and “controversial.”Orange-bar lengths represent uncertainty scores, helping users scan contexts and focus on those with high scores.Source: Uncertainties · p. 11, Figure 4 · p. 11
• Figure 5 presents an interactive concept tree of concepts co-occurring with “vaccine” or “vaccination,” which users can inspect alongside underlying citation contexts.The tree shows the hierarchical representation of those co-occurring concepts.Source: Uncertainties · p. 11, Figure 5 · p. 12
• The section then introduces Structural Variation Analysis as a predictive method for identifying newly published articles with potential to transform their underlying literature.SVA aims to identify articles that may have profound future impacts.Source: Uncertainties · p. 12

Structural Variation Analysis (SVA)

• SVA compares each new article’s co-occurring links against the network of literature published before it, assigning transformative-potential scores to novel links between distinct clusters.The method extends Structural Hole Theory (Burt 2004, Chen 2012) to scholarly publication networks.Source: Structural Variation Analysis (SVA) · pp. 12–13
• Epistemic uncertainty is unevenly distributed: cluster #0 has the most nodes with significant uncertainty, two of the three most uncertain nodes are in #2, and #6 has low overall uncertainty.The separated view also places Li, Guan et al. (2020) in cluster #0.Source: Structural Variation Analysis (SVA) · pp. 13–14, Figure 7 · p. 14
• A newly published article scored high for transformative potential on centrality divergence and also ranked high on modularity change, with references distributed across clusters.Figure 9 illustrates the article’s cross-cluster references.Source: Structural Variation Analysis (SVA) · p. 15, Figure 9 · p. 16
• For Fang and Meng (2020), citation contexts for four references appeared in distinct topical sections, partly explaining the diverse footprint SVA detected.Contexts describe remdesivir findings—about 68% clinical improvement with compassionate use [14], versus no significant clinical benefit in a randomized trial [15]—and cytokine storm roles [54,55]. MAG lacked these contexts, which were located manually.Source: Structural Variation Analysis (SVA) · pp. 16–17, Table 4 · p. 16
• About 10% of dataset records had citation contexts; nearly 15% of those mentioned vaccines, while SVA’s strongest modularity-change list aligned with citation-based ranking.Most listed articles were published in June 2020; the second-ranked had 19 citations. The authors suggest revisiting the list as contexts become available to assess whether transformative potential is realized or fades.Source: Structural Variation Analysis (SVA) · p. 17, Table 5 · p. 18

• The workflow supports monthly-scale literature studies and user-built datasets, while enabling DCA with MAG as with established, more selective sources such as Web of Science.The authors describe comparison across bibliographic databases as a new possibility for researchers.Source: Discussion · p. 18
• Integrating citation contexts with visual analytics links structural indicators to linguistic uncertainty cues, eases inspection of incubation-period discussions, and supports tracking potentially transformative articles.The interactive visualization reduces overhead and cognitive load.Source: Discussion · p. 18
• An equivalent topic search returned 29,858 Web of Science records versus 130,000 in CORD-19; MAG was in the same 70,000–90,000-record range as Lens and Dimensions.Source: Discussion · p. 18
• About 10% of dataset records had citation contexts; they clustered in the larger network’s core, while records without contexts tended toward its periphery.Despite this coverage limit, the available contexts revealed answers to specific questions, including literature consensus on the mean incubation period.Source: Discussion · p. 19
• MAS offers free MAG API access with call limits, while hosting a local MAG copy can remove those restrictions; the authors expect MAS and MAG data quality to improve.Source: Discussion · p. 19

• The method lets researchers tailor dataset breadth and depth, then apply existing tools to their chosen topics at their own pace.It is presented as addressing significant shortcomings in scientometric studies of literature.Source: Conclusion · p. 19
• Easy access to citation contexts contributes to visual analytic workflows and may help enable a fully integrated study of scientific literature.The authors describe integrating citation contexts with network-centric analysis as a stepping stone to studying literature at multiple levels of detail, including patterns and trends.Source: Conclusion · p. 19