Source-linked AI summary
A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments
Ciriaco Andrea D'Angelo, Cristiano Giuffrida, Giovanni Abramo
TL;DR
Large-scale bibliometric assessment is hindered by the difficulty of identifying the true authors of publications. The paper presents a heuristic process integrating external structured data, generating candidate mappings, and filtering false positives; its precision and recall appear adequate for typical bibliometric assessments, with practical scalability and expandability advantages.
Problem
Peer review cannot measure productivity comprehensively, while large-scale bibliometrics faces severe author-identification problems caused by ambiguous affiliations and homonyms.
Method
The paper integrates an external database of researcher identities with bibliometric records, generates candidate mappings, and filters false positives using data-driven heuristics.
Results
The proposed algorithm offers practical implementation, scalability, and expandability advantages, with precision of 95.6% and recall of 93.8% in a sample test.
Takeaways & Limitations
The approach enabled construction of a national bibliometric database supporting large-scale research evaluation at the level of individual scientists.
Takeaways & Limitations
Perfect homonyms are not further disambiguated and represented 0.043% of researchers in the chosen Italian dataset.
Abstract
from arXiv · showhide
National exercises for the evaluation of research activity by universities are becoming regular practice in ever more countries. These exercises have mainly been conducted through the application of peer-review methods. Bibliometrics has not been able to offer a valid large-scale alternative because of almost overwhelming difficulties in identifying the true author of each publication. We will address this problem by presenting a heuristic approach to author name disambiguation in bibliometric datasets for large-scale research assessments. The application proposed concerns the Italian university system, consisting of 80 universities and a research staff of over 60,000 scientists. The key advantage of the proposed approach is the ease of implementation. The algorithms are of practical application and have considerably better scalability and expandability properties than state-of-the-art unsupervised approaches. Moreover, the performance in terms of precision and recall, which can be further improved, seems thoroughly adequate for the typical needs of large-scale bibliometric research assessments.
Keywords
The paper concerns research assessment, authorship, author name disambiguation, bibliometrics, and universities.
- The paper addresses research assessment and authorship in bibliometric contexts.
- Author name disambiguation is identified as a central topic.
- The cited article is published in the Journal of the American Society for Information Science and Technology, volume 62, issue 2, pages 257–269.
Introduction
The paper motivates large-scale bibliometric research assessment by identifying limitations of peer review and author disambiguation, then proposes a structured-data heuristic approach designed for practical, scalable use.
- Motivation: Peer review evaluates only a limited subset of research products, undermining measurement robustness and validity and preventing productivity measurement.
- Motivation: Large-scale bibliometric assessment requires identifying individual scientists’ outputs without relying on institutional submission of publication data.
- The disambiguation problem: Ambiguous affiliations and numerous homonyms make author identification exceptionally difficult in large scientist populations.
- Prior limitations: Existing supervised approaches require costly training sets, while standard pairwise methods can mishandle publications outside an author’s main work.
- Proposed approach: The proposed approach integrates external structured data, generates candidate author-identity mappings, and filters false positives with data-driven heuristics.
- Practical properties: The approach is presented as easy to implement, scalable, expandable, linear in processed tuples, and less exposed to transitivity problems through global metrics.
Database integration
Database integration combines bibliometric records with an external authority database, after which aggressive matching generates candidate author-identity mappings.
- Database integration: The first phase integrates the bibliometric database with an external database of Italian university researchers and their attributes.
- Database integration: The external source excludes researchers without formal university positions, non-university research scientists, and foreign-based coauthors.
- Database integration: Applicability outside Italy depends on access to an authority external database, which should be readily available in countries with public university systems.
- Mapping generation: Mapping generation produces a superset of author-identity pairs that should contain the real mapping, with at most one true identity per author.
- Mapping generation: Aggressive matching explores all possible candidate identities, accommodating different name forms and matching criteria.
- Name variation: Compound surnames create multiple author-name forms, including cases where one surname may be indexed as a first name.
- Identity definition: External identities are defined annually without correlation across years, making automated longitudinal identity tracing complicated.
Filtering
Filtering removes false-positive author-identity pairs produced during mapping by organizing candidate pairs into author-specific clusters and accounting for homonym types.
- Filtering: Mapping produces false positives corresponding to possible homonyms, whose identifiable forms depend on the available database information.
- Homonym types: External homonyms involve a non-faculty or differently affiliated author sharing an identity with a faculty researcher.
- Homonym types: Inter-address homonyms occur when a faculty author has a homonym identity at another university.
- Homonym types: Intra-address homonyms occur within one university when homonym identities belong to different scientific disciplinary sectors.
- Homonym types: Perfect homonyms share the same university and scientific disciplinary sector, so the two identities become amalgamated rather than further disambiguated.
- Filtering limitation: Perfect homonyms represented 0.043% of researchers in the Italian university system and were considered negligible in the chosen dataset.
- Filtering: Candidate pairs for each author form a cluster, with singleton clusters representing authors free of identified homonyms or unresolved external homonyms.
The address filter
The address filter removes author–identity pairs whose bibliometric address is incompatible with the identity’s university affiliation. It uses matching rules and archives surviving non-empty clusters for later filters.
- The address filter: The address filter eliminates author–identity pairs with incompatible author and identity affiliations.The affiliations come from the bibliometric record’s address field and an external researcher database.
- The address filter: Its effectiveness depends on matching affiliations that are typically represented in different formats.
- The address filter: The method combines high filter performance with manual compilation and maintenance of a controlled vocabulary.The vocabulary contains over 30,000 rules matching over 89% of “Italy” addresses in the 2001–2007 I-NCR.
- The address filter: The controlled-vocabulary approach has good expandability because affiliations vary little over time compared with articles and authors.
- The address filter: After the address filter, all clusters with non-zero cardinality are archived for subsequent filtering.
The shared SDS filter
The heuristic procedure combines disciplinary matching, cluster-level filtering, and manual validation to resolve author identities. Reported tests show high precision and recall, while errors arise from both source data and filtering assumptions.
- The shared SDS filter: The maximum correspondence filter retains the single identity in each unresolved cluster whose SDS best corresponds to the article’s subject category.
- The shared SDS filter: The first address filter was the most aggressive, eliminating 65.6% of identities and 17.9% of publications relative to mapping-generation results.
- The shared SDS filter: The algorithm’s final validation test identified 28 false positives and 40 false negatives, yielding 95.6% precision, 93.8% recall, and a 94.7% f-measure.
- The shared SDS filter: 40% of identified false negatives arose from incorrect author affiliations, while 30% resulted from the WoS-SDS filter and 10% from address-matching vocabulary errors.
- The shared SDS filter: 96.2% true recall exceeded 95.6% precision after excluding false negatives caused by author or external-database errors.The adjusted counts were 24 false negatives and 28 false positives.
- The shared SDS filter: The proposed algorithm is presented as easy to apply, scalable, expandable, and adequate for typical large-scale bibliometric assessments.