Source-linked AI summary
New data, new possibilities: Exploring the insides of Altmetric.com
Nicolás Robinson-García, Daniel Torres-Salinas, Zohreh Zahedi, Rodrigo Costas
TL;DR
Citation-based evaluation has recognized limitations, motivating examination of whether Altmetric.com provides broad and meaningful alternative data. The paper analyzes DOI-bearing Web of Science publications through the Altmetric API, finding concentrated coverage across a small subset of sources and identifying remaining exhaustiveness and source-selection concerns.
Problem
The paper addresses uncertainty about Altmetric.com’s data richness, coverage, source selection, and usefulness for developing altmetric indicators.
Method
The authors match DOI-bearing Web of Science publications from 2011–2013 with the Altmetric API and analyze the retrieved records and their source structures.
Results
Five of 16 sources cover 95.5% of publications with altmetrics, with Twitter covering 87.1% and Mendeley 64.8%.
Takeaways & Limitations
Altmetric.com provides rich, transparent data, but Twitter and Mendeley dominate its coverage and therefore merit particular attention when interpreting its indicators.
Takeaways & Limitations
The authors identify language bias in blog and news retrieval and question whether Altmetric.com’s selected sources adequately represent diverse communities and audiences.
Abstract
from arXiv · showhide
This paper analyzes Altmetric.com, one of the most important altmetric data providers currently used. We have analyzed a set of publications with DOI number indexed in the Web of Science during the period 2011-2013 and collected their data with the Altmetric API. 19% of the original set of papers was retrieved from Altmetric.com including some altmetric data. We identified 16 different social media sources from which Altmetric.com retrieves data. However five of them cover 95.5% of the total set. Twitter (87.1%) and Mendeley (64.8%) have the highest coverage. We conclude that Altmetric.com is a transparent, rich and accurate tool for altmetric data. Nevertheless, there are still potential limitations on its exhaustiveness as well as on the selection of social media sources that need further research.
Introduction
Altmetrics emerged as proposed complements to citation-based evaluation, but their meaning, coverage, and source suitability remained insufficiently established. This paper therefore examines Altmetric.com’s data contents and potential as an altmetric-indicator source.
- Introduction: Altmetric indicators may broaden and accelerate assessment beyond citation counting, but these claims remained under-developed and required further study.Prior concerns included citation biases, publication delays, weak citation correlations, and uncertainty about whether altmetrics capture complementary aspects of impact.
- Introduction: Altmetric.com aggregates weighted impact information from multiple social-media sources and disambiguates links to publications across identifiers and sites.The provider unifies links to PubMed records, arXiv identifiers, DOI numbers, and publisher websites.
- Introduction: The study explores Altmetric.com’s contents and potential for developing altmetric indicators through a comprehensive, practical analysis of its available data.The stated aim is to answer which sources are included, how they are structured, and how extensively they cover publication impact.
1. Which data sources are included in Altmetric.com and how are they structured?
The paper asks which sources Altmetric.com includes, how its data are structured, and how widely those sources cover publication impact. It addresses these questions through practical extraction and detailed analysis of the tool’s data.
- 1. Which data sources are included in Altmetric.com and how are they structured?: The study examines which data sources Altmetric.com includes, how they are structured, and which sources cover more publication impact.These questions define the paper’s investigation of Altmetric.com as an altmetric-data source.
- 1. Which data sources are included in Altmetric.com and how are they structured?: The authors performed a practical extraction from Altmetric.com and conducted a detailed analysis of the data it provides.
Material and methods
The study matched DOI-bearing Web of Science publications from 2011–2013 against the Altmetric API and analyzed the records’ disaggregated fields, histories, users, and posts. The resulting records organize bibliographic, metric, demographic, and event-level information for each paper.
- Material and methods: 2,792,706 DOI numbers were matched with the Altmetric API, yielding 516,150 records and roughly 19% of DOI-bearing publications with social-media attention.The source set comprised Web of Science publications from 2011–2013, and the authors note DOI errors and incomplete DOI information in Altmetric.com.
- Material and methods: Altmetric.com records counts for readers, Facebook, blogs, news, Pinterest, Reddit, Twitter, video, LinkedIn, and other providers.The extracted structures include user counts, user lists, post counts, citation metadata, demographics, and provider-specific posts.
- Material and methods: Five record sections provide source scores, paper metadata, temporal score evolution, demographics, and detailed recorded fields.The record includes bibliographic details and alternative links, journal-score comparisons, country and public-type information, and the full set of recorded fields.
Description of sources collected by Altmetric.com
Altmetric.com identifies 16 sources and records multiple metric types, historical activity, and detailed mention data. Its coverage combines link recognition with limited text mining, while API records and the bookmarklet can differ in displayed fields and values.
- Description of sources collected by Altmetric.com: 16 sources are identified, with historical metrics and provider-specific fields retrieved through link recognition and, for blogs and news, text mining.Text mining is applied only to English-language sources and complements direct-link matching for blogs and news.
- Description of sources collected by Altmetric.com: Discussions and mentions are the most common metric types, followed by readership counts from Mendeley, Connotea, and Citeulike.Other collected metrics include videos, reviews, and Question and Answer discussion threads.
- Description of sources collected by Altmetric.com: The sources are mostly Web 2.0 services, with some provider usage concentrated in particular fields such as biomedical research or computer and natural sciences.Research Highlights is the stated exception, because it includes citation data from Nature’s highlights section.
- Description of sources collected by Altmetric.com: The bookmarklet does not display every API element, including all tweets and retweets, and occasional discrepancies occur between source records such as Question and Answer threads.Except for Misc, the identified fields contribute to each paper’s aggregated Altmetric score.
Coverage of Altmetric.com for WoS publications with DOI in 2011-20134
Altmetric.com covered 19% of the original Web of Science DOI sample, with coverage concentrated in a few social media sources. Twitter and Mendeley were the most widely represented, while five sources accounted for 95.5% of retrieved papers.
- 19% of the original Web of Science DOI publication set was included in Altmetric.com with some altmetric impact.
- Twitter covered 87.1% of papers in Altmetric.com, followed by Mendeley at 64.8%.
- No other social media source exceeded 20% coverage, although Facebook reached 19.9% of included papers.
- Five sources—Twitter, Mendeley, Facebook, Citeulike, and blogs—were present in 95.5% of papers retrieved from Altmetric.com.
- Table 3 organizes the five leading sources by papers, total counts, unique users, and percentage of Web of Science papers.
Discussion and concluding remarks
Altmetric.com provides rich, transparent data, but its coverage and source selection constrain how comprehensively altmetric impact can be studied. The study highlights concentration among a few sources and calls for more rigorous validation and source selection.
- Mention retrieval remains difficult to assess for exhaustiveness, precision, and correctness, particularly when blogs and news media require text-mining and link recognition.
- English-only text mining for some sources creates language bias, while omitted platforms and scientific networks may limit coverage across communities and disciplines.
- Source selection is a serious limitation because covered platforms may reflect practical considerations rather than a fully justified conceptual or theoretical framework.
- Five of Altmetric.com’s 16 sources account for 95.5% of publications with altmetrics, concentrating coverage among a small group of social media sources.Twitter and Mendeley are the most prominent sources, but their coverage differs across publications and recorded activity.
- Twitter covers more publications, whereas Mendeley records higher counts and users, making both sources especially relevant for studying altmetric impact.
- Altmetric.com records counts, mentions, user information, origins, and dates permanently, supporting reproducibility, retrospective analysis, and analyses beyond simple mention counts.
- Altmetric.com is a relevant open provider with high-quality, transparent, and rich data, but its retrieval methods and source selection require further research.
Notes
The notes identify related tools, incomplete publication-year coverage, and the JSON format referenced for the study’s data representation.
- ImpactStory was founded in 2011 by Jason Priem and Heather Piwowar and was originally called Total-Impact.
- Plum Analytics was founded in late 2011 and was later acquired by EBSCO Publishing.
- Altmetric.com was founded by Euan Audie in 2011 and became one of the main altmetric providers.
- The 2013 publication year was incomplete because only one third of publications had been uploaded, but this did not affect the descriptive coverage analysis.
- The paper directs readers to an external reference for more information about the JSON format.