How are new citation-based journal indicators adding to the bibliometric toolbox?

Introduction

• The Impact Factor measures the impact of an average journal item, not the impact or influence of the journal as a whole.Inferring journal-wide quality from an average item may commit an ecological fallacy.Source: Introduction · p. 2
• Influence combines impact with productivity, so journal size matters alongside the impact of the average item.Productivity can be indicated by total documents, citations, or publications.Source: Introduction · pp. 2–3
• The Immediacy Index counts citations in the publication year, while cited half-life is the median age of articles cited in the current JCR year.The Impact Factor is the average number of citations in a given year to citable items from the two preceding years.Source: Introduction · pp. 2–3
• Prior analysis found two dimensions among ISI indicators—size and impact—and usage data correlated with size in the 1993 dataset.Total Cites correlated more strongly with faculty ratings and library use than Impact Factor did in the cited validation study.Source: Introduction · pp. 3–4
• The paper compares newer indicators including H-index, SJR, and PageRank with older indicators; Scopus, ISI databases, and Google Scholar provide data sources.The H-index is described as reflecting publication count and citations per publication; SJR is presented as a Scopus-domain counterpart to the Impact Factor.Source: Introduction · pp. 4–6
• The study asks whether indicators capture a common dimension, how they relate to measures of impact, influence, and size, and how stable they are over time.It compares 2007 data with 2006 and extends the analysis with social-network centrality measures.Source: Introduction · p. 7

Methods and materials

• The data combine Science Citation Index and Social Science Citation Index journals, yielding 7,940 journals in 2007 and 7,611 in 2006.The combined datasets correct for overlap between the databases.Source: Methods and materials · pp. 7–8
• A full citation matrix represents journals as a valued, directed graph, from which cited- and citing-dimension degree, betweenness, and closeness measures can be computed.The matrix can also provide journal PageRanks.Source: Methods and materials · p. 8
• The study uses Scopus Total Cites for a three-year period as its best available proxy for Total Cites in that database.The measure includes citations to all document types and references to documents published in the preceding three years.Source: Methods and materials · pp. 8–9
• Scopus H-index values were updated across download dates and retrospectively across years, making the reported year-specific values download-dependent.Different values were found between April and June 2008 searches.Source: Methods and materials · p. 9
• SJR values changed across downloads for earlier years, and the study reports that the reason for these retrospective updates was unexplained.The analysis used 2006 values downloaded in April 2008 and 2007 values downloaded in June 2008, assuming annual database reloads.Source: Methods and materials · pp. 9–10
• For 2007, the study uses factor and correlation analyses, then adds centrality measures to examine how they contribute to understanding the indicators.The same analysis is also performed for 2006.Source: Methods and materials · pp. 10–11
• The indicator structure was virtually similar across 2006 and 2007, suggesting stability; temporal autocorrelations were then calculated for matching journal values.The dynamic analysis covered 5,861 journals present in both databases in both years.Source: Methods and materials · p. 11

Journal indicators

• PageRank rankings remain strongly correlated across damping factors d = 0.15 and d = 0.85, with Pearson’s r = 0.942 and Spearman’s ρ = 0.934.The damping-factor choice made no difference to the factor analysis.Source: Journal indicators · p. 12
• The H-index combines size and impact more than any other indicator, while PageRank Centrality is a size indicator.The authors suggest the H-index’s combination of dimensions may explain its rapid popularity among evaluators.Source: Journal indicators · p. 14
• The number of references and number of documents are highly correlated and indicate size; Scopus and ISI Total Cited correlate at r = 0.94.The Total Cited correlation holds despite different database definitions; restricting Scopus citations to the last three years shifts its loading somewhat toward impact.Source: Journal indicators · p. 14
• SJR correlates with the Impact Factor and Immediacy Index, but is more orthogonal to size than the Impact Factor and normalizes for size more strongly.The Impact Factor correlates less with SJR than with the Immediacy Index.Source: Journal indicators · pp. 14–15, Table 3 · p. 15
• Comparing 2006 with 2007 shows robust two-factor solutions, while the Audience Factor does not add structurally to the existing indicators.The Audience Factor comparison used 4,277 journals.Source: Journal indicators · pp. 15–16

Network indicators

• The five-factor solution explains 85.7% of common variance and adds third and fourth dimensions through the network indicators, while Cited Half-life remains a fifth factor.The screeplot indicates that the five-factor solution should be considered.Source: Network indicators · p. 17
• Indegree correlates highly with the H-index (r = 0.89), and the H-index and Indegree share interfactorial complexity with PageRank.Indegree is described as a measure of reach: the number of citation relations, without weighting for the number of citations on those relations.Source: Network indicators · p. 19
• PageRank correlates negatively with the impact dimension and has interfactorial complexity with H-index and Indegree.The only journal indicator sensitive to adding network indicators is PageRank, because it is based on graph theory.Source: Network indicators · pp. 18–19
• Cited Half-life correlates neither positively nor negatively with network indicators, consistent with its focus on time while network indicators are momentary.A historical dimension can be programmed into the network approach, but must be added explicitly.Source: Network indicators · p. 20

Stability and change

• SJR and H-index are more stable across 2006–2007 than traditional ISI indicators (r > 0.99).The comparison uses 5,861 journals included in both databases in both years.Source: Stability and change · pp. 20–21
• Indicators in the citing dimension are less stable than those in the cited dimension, as citing introduces change into the database.Source: Stability and change · p. 21
• Closeness centrality is less stable than the other network indicators (0.61 < r < 0.79).Source: Stability and change · p. 21

• SJR and H-index avoid arithmetic-mean normalization, an advantage for heavily skewed citation data.Source: Conclusions · p. 22
• The Impact Factor measures the average quality of articles, not journal quality; treating it as a journal-level measure risks an ecological fallacy, especially for heterogeneous journals.Source: Conclusions · pp. 22–23
• PageRank was not an impact indicator, while SJR, conceptually derived from PageRank, appears to offer an alternative to the Impact Factor.Source: Conclusions · p. 23
• Cited Half-life distinguishes expected citation behavior across document types independently of disciplinary differences.Source: Conclusions · p. 23
• Impact-axis indicators, including Audience Factor, are highly correlated.Source: Conclusions · p. 23
• Although the H-index is easy to retrieve and remember, combining size and impact can oversimplify distinct citation profiles and produce counter-intuitive ties.For example, an author with three papers cited 100 times each can rank equally with one whose top three papers have only marginally more than three citations each.Source: Conclusions · pp. 22, 24