Source-linked AI summary
A review of the characteristics of 108 author-level bibliometric indicators
Lorna Wildgaard, Jesper W. Schneider, Birger Larsen
TL;DR
The widespread use of author-level bibliometric indicators has expanded the need to understand what they reflect and how usable they are. This review examines 108 indicators, finding that 79 are potentially useful to end-users while no single indicator captures overall researcher impact.
Problem
Individual-level indicators are increasingly used, but single indicators cannot capture overall impact and their limitations require comparison.
Method
The review examined 108 author-level indicators, assessing their intended use, data requirements, citation-data availability, and mathematical complexity.
Results
79/108 indicators scored ≤3 for both data collection and calculation and were judged potentially useful for end-users.
Takeaways & Limitations
Only combinations of indicators can approximate overall researcher impact, so users should avoid representing publication effects with a single numerical value.
Takeaways & Limitations
H-type indicators can produce misleading results before a scientist reaches a certain level of scientific maturity, generally above 50 papers.
Abstract
from arXiv · showhide
An increasing demand for bibliometric assessment of individuals has led to a growth of new bibliometric indicators as well as new variants or combinations of established ones. The aim of this review is to contribute with objective facts about the usefulness of bibliometric indicators of the effects of publication activity at the individual level. This paper reviews 108 indicators that can potentially be used to measure performance on the individual author level, and examines the complexity of their calculations in relation to what they are supposed to reflect and ease of end-user application.
Introduction
The review examines how author-level bibliometric indicators reflect publication effects, how complex they are to calculate, and whether end-users can apply them. It responds to widespread use of many indicators by comparing their intended meanings, data requirements, limitations, and usability.
- Motivation: Individual assessment increasingly relies on accessible author-level indicators, but more than one hundred variants manipulate a small set of variables in different ways.The review addresses their use by administrators, evaluators, and researchers.
- Motivation: Performance is only a proxy for impact and cannot be captured by a single bibliometric indicator.The h index is often treated as indicating both quality and impact despite this limitation.
- Scope: The authors review indicators of publication effects, excluding activities such as societal impact, web presence, leadership, teaching, and innovation.The scope is limited to traditional publication effects within the academic community or public sphere.
- Research questions: The review asks which author-level indicators end-users can calculate and whether their intended meanings can be understood.These questions connect practical usability with interpretability.
- Method: The analysis investigates each indicator’s intended use, calculation, and data requirements, scoring citation-data availability and mathematical intricacy on five-point scales.Indicators scoring 4 or above on either complexity criterion are judged too complex for end-users.
- Method: The review identifies 108 indicators and presents their calculations, purposes, limitations, advantages, complexity scores, and functionality in supplementary tables.Its framework extends a citation-and-publication model from research-group assessment to author-level assessment.
Overview of the identified indicators
The review identifies 108 author-level indicators across publication counts, journal impact, citation effects, and citation-normalized measures, evaluating their intended meaning, limitations, and complexity for end users. Most indicators were judged potentially usable, while more complex measures require specialized software or advanced calculations.
- Overview of the identified indicators: 35/108 indicators are adaptations of the h index, comprising one third of the identified author-level indicators.The review presents category totals and complexity assessments for all 108 indicators.
- Overview of the identified indicators: 79/108 indicators scored ≤3 for both data collection and calculation and were judged potentially useful for end users.The remaining 29 indicators scored ≥4 in either data collection or calculation.
- Publication Count, category 1: 15 publication-count indicators all scored ≤2 in complexity and use simple counting or ratio models with equal or fractional author contribution.Some counts restrict publications to predetermined databases, journals, publishers, or publication types.
- Qualifying output as Journal Impact, category 2: Journal-impact indicators are applied at author level to suggest the visibility of a researcher’s work, although they were originally designed for journals or groups.JIF is commonly used to measure the impact factor of journals in which an author has published; SNIP normalizes citation practices between fields.
- Effect of Output, category 3: Citation-effect indicators include whole or fractional citation counts, self-citation adjustments, uncited-paper measures, and indicators of highly visible papers.Nine of the 11 identified citation-counting indicators were judged useful in assessment, and fractional approaches adjust for multi-authorship.
- Effect of Output, category 3: Field- and portfolio-normalized indicators compare citations with expected field performance or publication volume, but database coverage and benchmark choices constrain interpretation.IQP uses user-defined field averages and expected-to-actual citation ratios; citation-based ratios can differ across databases.
Indicators that rank the publications in the researcher’s portfolio, category 4
Category 4 covers indicators that rank publications within a researcher’s portfolio, including h-dependent, h-independent, field-adjusted, and co-authorship-adjusted measures. These indicators differ in what citation patterns they emphasize and in their calculation complexity.
- h-dependent indicators: The h index and related h-dependent indicators rank portfolio publications through cumulative citation measures, with 10 of 16 scoring ≤3 for calculation complexity.The listed variants include h, m, e, hmx, Hg, h2, A, R, ħ, and Q2.
- h-dependent indicators: Several h-dependent variants adjust or extend h by accounting for excess, average, median, square-root, geometric-mean, or cross-database citation effects.e measures excess citations in the h-core; A and m use average and median citations, while R, hg, h2, and Q2 apply other transformations.
- h-independent indicators: The h-independent group includes additional variants such as R, Q2, h per decade, citation-weighted h, tapered h, and rational h indicators.These indicators are listed as alternatives for ranking cumulative impact in the portfolio.
- h-independent indicators: Six h-independent cumulative-impact indicators were identified, and four—w, f, g, and t—scored ≤3 in complexity.The w index provides a simple prestige measure, while the g index weights highly cited papers more than h does.
- h adjusted to field: Field-adjusted indicators aim to account for differing publication and citation habits, with normalized h and n proposed for comparisons across fields.Normalized h divides h by articles outside the h-core, while n divides a researcher’s h by the highest h in the major field’s journals.
Impact over time, category 5
The review distinguishes indicators of impact over time normalized to a researcher’s portfolio from those adjusted to the field. Portfolio-normalized indicators include several potentially useful measures, whereas field-normalized indicators are sophisticated but generally too complex for end-users.
- Impact over time: 12 indicators of impact over time were identified; 6 were judged potentially useful with complexity ≤3.Ten compare impact over time relative to the portfolio, while two compare it with the field’s expected aging rate.
- Portfolio-normalized indicators: Eight portfolio-normalized indicators include age-weighted citation rates, AR, m and mg quotients, Price Index, and citation age c(t).Most are ratio-based models; AR uses the square root of average annual citations in the h-core, and the m quotient is also h-related.
- Portfolio-normalized indicators: AWCR sums citations across a researcher’s work after dividing each paper’s citations by its age, while AW is the square root of AWCR.Per-author AWCR additionally normalizes for the number of authors per paper.
- Portfolio-normalized indicators: The Price Index measures the percentage of citations received within five years of publication, while c(t) indicates citation age.Adjusting citation rates for changes in the citing population or discipline requires a corrective factor.
- Field-normalized indicators: Field-normalized impact-over-time indicators were judged too complex for end-users, despite addressing field and document-type differences.Examples include Classification of Durability, a(t), and Age and Productivity; the latter requires demanding data collection, multiple indicators, and Web of Science journal categories.
- Practical use: The review concludes that indicator availability and data accessibility limit practical application, supporting recommendations of indicator groups rather than single measures.The broader assessment also notes that publication counts can distort output scope and journal indicators can reflect popularity rather than prestige.
3 The IQP calculator can be downloaded from: http://tinyurl.com/nj7s834
The review finds that author-level indicators vary in stability, fairness, interpretability, and calculation complexity. No single indicator captures overall researcher impact; combinations require knowledge of what each measures and how it is calculated.
- h-dependent indicators, 4a: The h index is widely used because its simplicity and recognisability outweigh concerns about representativeness.For researchers with fewer than about 50 papers, stability issues can produce misleading results.
- h-dependent indicators, 4a: h-type indicators can be biased across fields and toward senior researchers, while also losing citation information and disadvantaging selective publication or career-break patterns.The review also notes criticisms concerning publication behavior and granularity.
- h adjusted to field, 4c: Field-adjusted indicators address obstacles to fair cross-field assessment but can have severe limitations despite being simple to calculate.The n index divides h by the highest h index of the major journal in which the researcher publishes.
- h corrected for co-authorship, 4d: Co-authorship adjustments lack an agreed correct distribution of credit, and some methods can discourage collaboration or become sensitive to extreme values.Reducing the h-core can increase sensitivity to extreme values, whereas larger h-cores affect precision.
- Conclusions: The review concludes that indicators only partially capture impact, and combining indicators is necessary to approximate a researcher’s overall impact.There is no agreement on which combination best expresses impact or fits every assessment purpose.
Qualifying output as Journal Impact, category 2
Journal-impact indicators are often applied to suggest an author’s visibility even though many were designed for journals or groups. Their usefulness depends on the comparison context, data source, field, and calculation burden.
- Journal impact indicators: SIF is described as robust for permanent impact because it mixes publications from different years.It is readily available for end-users.
- Journal impact indicators: Source-normalized indicators correct for differences in citation practices between fields, but source normalization does not correct for literature-growth differences or unidirectional citation flows.SNIP was revised to correct counterintuitive properties and does not require explicitly defined field boundaries.
- Journal impact indicators: The immediacy index reflects how quickly articles are cited, but frequently issued journals may advantage articles published earlier in the year.Its interpretation is also influenced by journal history, prestige, and atypical references.
- Journal impact indicators: Journal indicators can support within-discipline comparisons, while cross-field comparisons remain constrained by disciplinary differences, database coverage, and classification choices.The Article Influence Score retains large disciplinary differences, and article classification can produce very different evaluation ratings.
- Journal impact indicators: Journal impact factors can be applied to individual researchers, but variation among articles within one journal makes this use vulnerable to misuse.The paper notes that journal-level measures were not originally designed to assess individuals.
- Journal impact indicators: Diachronous IF can better represent a researcher than SIF, but it requires more resources because data must be collected manually.Its calculation uses citations from multiple citing years to documents from a fixed publication year.
Effect of output as citations, category 3a
Citation-count indicators range from simple totals and uncited-publication counts to fractional and self-citation measures. Their interpretation is constrained by database coverage, field differences, citation distributions, and uncertainty about author credit.
- Citation counts: Citation counts depend on the database used, and sophisticated indicators often cannot be compared across sources such as WoS, Scopus, and Google Scholar.The scope, validity, reliability, and cost of citation collection depend on the selected citation index.
- Citation counts: The i10-index counts publications with at least 10 citations, but the threshold’s meaning is highly field dependent.Google Scholar data require careful verification because documents and cited references may be misattributed.
- Citation counts: Citation counts do not capture citation quality or timeliness, and self-citations can be especially problematic in small assessments.A citation’s origin and whether it is positive or negative are not represented by basic counts.
- Citation counts: Self-citations are difficult to define consistently and can affect reliability and validity when assessments use small amounts of data.Possible definitions include citations to oneself, a co-author, or an institutional colleague.
- Citation counts: Fractional citation counting gives an author c/m credit for an m-authored paper receiving c citations, distributing credit evenly among authors.The approach is designed to reduce dependence on co-authorship and support fairer comparisons.
Effect as citations normalized to publications and field, 3b
Normalized indicators adjust citation-based performance for field, publication type, age, productivity, or citation distribution. The review presents normalization as useful but sensitive to benchmarks, classification, skewness, time windows, and data scope.
- Normalization: Percentile-based measures can prevent a single highly cited publication from receiving excessive weight and can reveal whether a high normalized score reflects a few papers or broad citation performance.These measures identify representation among highly cited publications at specified thresholds.
- Normalization: Top-publication comparisons become less accurate across fields and time as the representation level increases.The degree to which top n% publications are over- or under-represented differs across fields and over time.
- Normalization: IQP corrects for academic age, field averages, and the ratio of expected to actual citations, producing measures of above-average papers and citation performance.It has been tested in natural sciences, medicine, and psychology and depends on Web of Science field-specific journal impact factors.
- Normalization: Percentiles are considered suitable for normalizing citation counts by subject, document type, and publication year.Unlike mean-based indicators, percentiles are not affected by skewed distributions.
Indicators that rank the portfolio: h dependent, category 4a
H-dependent indicators extend or modify the h-index to capture citation intensity, highly cited papers, citation distribution, career stage, and field-sensitive comparisons. Their usefulness is balanced against dependence on h, arbitrary parameters, database assumptions, and limitations in discrimination or interpretation.
- The h-index counts papers ranked by citations, with H papers each receiving at least H citations.
- The e-index captures excess citations in h-core papers that the h-index ignores.It can only be calculated after h is determined.
- The g-index gives greater weight to highly cited papers and can distinguish researchers with similar h-indices, but a single highly cited paper may disproportionately increase it.
- The a-index measures the average citations of h-core papers, so larger values indicate highly cited papers relative to the remainder.
- Hpd uses citations per decade and is nearly constant for mature scientists while slowly declining for inactive scientists.This supports comparison across scientists of different ages, but its scaling factor of 10 is arbitrary.
- The hα-index offers finer comparisons among researchers with the same h by varying α, but no agreed value exists and its sensitivity requires further study.Small α emphasizes publication quantity, whereas large α emphasizes the most cited paper.
Indicators that rank the portfolio: h independent, category 4b
H-independent indicators rank portfolios without directly depending on the h-index, emphasizing citation concentration, elite papers, or citation distributions. They provide alternative discrimination but can rely on arbitrary scaling, averages, or interpretations that limit comparability.
- The w-index defines w as the highest number of papers with at least 10w citations each.Values of 1–2, 3–4, and 10 are associated with progressively greater levels of scientific achievement in the cited interpretation.
- The g-index can grow disproportionately for a researcher with one highly cited paper and a mediocre citation core.
- The f-index uses fractional counting and the harmonic mean to weight citations, making it more sensitive to small researcher differences than h- and g-indices.
- The t-index uses the geometric mean for a similar citation-weighting scheme, but geometric averaging places less weight on citation distribution.
- The π-index summarizes the citations received by the top square root of a researcher’s papers.Its value depends on citation rates within an elite set scaled by an arbitrary prefactor.
Indicators that rank the portfolio: h adjusted to field, category 4c
Field-adjusted indicators normalize citation performance to support comparisons across disciplines with different citation and publication practices. Their applicability is constrained by field-specific constants, reference choices, validation needs, and calculation complexity.
- The n-index divides h by the number of publications when h publications meet the citation threshold, thereby accounting for portfolio size.
- The normalized h-index can compare scientists of different scientific ages, but it must be used alongside the h-index and may reward less productive, highly cited authors.
- Article-level normalization divides each citation count by the average for its subject category, correcting citation rates for field variation.
- Field-normalized indicators rescale citation counts or ranks using discipline-dependent constants to compare researchers across fields.The constants are not available for all fields.
- Field-adjusted measures may be difficult to calculate, depend on journal-based or impact-factor references, and remain awaiting validation.
- The x-index divides a researcher’s absolute score by a reference score to describe productive-core quantity and quality relative to peers.
Indicators that rank the portfolio: h corrected for co-authorship, category 4d
Co-authorship-adjusted indicators modify h-type measures by fractionalizing papers, citations, author counts, or author position. They aim to estimate individual contribution more fairly, but results depend on credit-allocation choices and can be difficult to calculate.
- The alternative h-index divides h by the mean number of authors in h publications to approximate the papers a researcher would have written alone.The mean is sensitive to extreme author counts and can penalize papers with many authors.
- The pure h-index uses author counts and relative byline rank to correct individual h-scores for co-authorship.
- Co-authorship-adjusted results vary with the method used to distribute author credit, and some variants are difficult to calculate.
- Co-authorship-adjusted indicators account for the number of collaborators, author rank, or fractional credit when evaluating individual impact.
- The hm-index uses inverse author counts to produce a reduced number of papers meeting the citation threshold, softening multi-author influence.
- Fractional paper counts can alter the h-core less than fractional citation counts and provide finer individual-score granularity.
Indicators of impact over time: normalized to portfolio, category 5a
These indicators adjust citation impact for publication age, citation age, career length, or time-dependent changes in performance. Their usefulness depends on calculation practicality, field assumptions, and how weighting choices affect authors.
- Usability: All indices require verified publication data and data from one or more citation databases.
- Age-weighted indicators: Using the sum over all papers allows younger, less cited publications to contribute to the AWCR.
- Age-weighted indicators: AWCR measures citations to an author’s entire body of work while adjusting for the age of each paper.The AW-index is its square root, and the per-author version additionally normalizes for the number of authors.
- Dynamic indicators: AR evaluates performance changes using the square root of summed average annual citations for articles in the h-core.It incorporates h-core size and contents, citation counts, and h-velocity.
- Career-length indicators: Career-normalized indices support comparisons across different career lengths, but first publication may not accurately mark career start and can disadvantage part-time researchers or career interruptions.The m-quotient also does not address h-index disadvantages concerning publication and citation quality.
- Dynamic indicators: Dynamic H-type indicators can distinguish researchers whose h-indices are currently rising from those whose h-indices remain unchanged.The approach uses the R-index and h-velocity, but its h-dependent velocity fitting is identified as a limitation.
Indicators of impact over time: normalized to field, category 5b
Field-normalized indicators require verified publication data and information from one or more citation databases. These data requirements apply across the indices covered here.
- Verified publication data are required for all indices.
- All indices require data from one or more citation databases.
- The data requirement applies across the reviewed indicators rather than to a single metric.