Source-linked AI summary
Indicators of the Interdisciplinarity of Journals: Diversity, Centrality, and Citations
Loet Leydesdorff, Ismael Rafols
TL;DR
A citation-based indicator for journal interdisciplinarity was lacking. The study compares network, distributional, and Rao-Stirling measures using citation data from 8,207 journals, finding that different indicators capture different aspects of interdisciplinarity while Rao-Stirling results depend strongly on the distance measure.
Problem
A specific citation-based indicator of journal interdisciplinarity had been lacking among available journal indicators.
Method
The study compares betweenness centrality, Shannon entropy, the Gini coefficient, and Rao-Stirling measures using journal citation distributions and networks.
Results
Different indicators may capture different understandings of the multifaceted concept of interdisciplinarity.
Takeaways & Limitations
The investigated indicator classes can all be considered possible candidates for an interdisciplinarity indicator.
Takeaways & Limitations
The study did not find a robust and unambiguous indicator of interdisciplinarity, and differing exponents complicated indicator identification.
Abstract
from arXiv · showhide
A citation-based indicator for interdisciplinarity has been missing hitherto among the set of available journal indicators. In this study, we investigate network indicators (betweenness centrality), journal indicators (Shannon entropy, the Gini coefficient), and more recently proposed Rao-Stirling measures for "interdisciplinarity." The latter index combines the statistics of both citation distributions of journals (vector-based) and distances in citation networks among journals (matrix-based). The effects of various normalizations are specified and measured using the matrix of 8,207 journals contained in the Journal Citation Reports of the (Social) Science Citation Index 2008. Betweenness centrality in symmetrical (1-mode) cosine-normalized networks provides an indicator outperforming betweenness in the asymmetrical (2-mode) citation network. Among the vector-based indicators, Shannon entropy performs better than the Gini coefficient, but is sensitive to size. Science and Nature, for example, are indicated at the top of the list. The new diversity measure provides reasonable results when (1 - cosine) is assumed as a measure for the distance, but results using Euclidean distances were difficult to interpret.
Introduction
The study addresses the lack of a citation-based journal indicator specifically focused on interdisciplinarity by comparing network, distributional, and diversity measures. It evaluates whether these approaches can provide useful journal-level indicators using aggregated citation relations.
- A citation-based indicator specifically focused on journal interdisciplinarity had been lacking, despite the concept’s policy relevance.
- Betweenness centrality is considered a candidate because journals that connect otherwise distinguishable citation clusters may bridge specialties.
- The study compares betweenness centrality, Shannon entropy, the Gini coefficient, and Rao-Stirling diversity as candidate indicators.
- The analysis asks whether different indicators can be benchmarked against one another and whether Rao-Stirling diversity is useful at the journal level.
- Rao-Stirling diversity combines citation-distribution variety with distances between journals in the citation network.
Data processing
The study assembled an 8,207-journal citation matrix from the Science and Social Sciences Citation Indexes and processed citing and cited dimensions separately. It aggregated rare citation relations and handled journals with only self-citations as a special case.
- Data processing: 8,207 journals were assembled from 6,598 Science Citation Index journals and 1,980 Social Sciences Citation Index journals, with 371 overlaps.The resulting asymmetrical matrix represented the same journals in cited and citing dimensions.
- Data processing: The asymmetrical 8,207-journal citation matrix was saved as a systems file for generating similarity and distance matrices.Betweenness centrality values were generated using Pajek2 and UCINet.
- Data processing: 8,159 journals were processed in the cited dimension and 8,064 in the citing dimension.These counts were reported for the 8,207-journal dataset.
- Data processing: Single citation occurrences were aggregated under “All other” on the citing side to reduce incidental noise and improve computational efficiency.This aggregation was applied to relations recorded from the citing side of the database.
- Data processing: 64 cited-dimension journals and 20 citing-dimension journals had only diagonal self-citations and therefore no Rao-Stirling diversity value.Their diagonal values were nevertheless included when computing Gini, probabilistic entropy, and centrality measures.
Gini coefficients and Shannon entropy
The vector-based indicators characterize interdisciplinarity through citation-frequency distributions: Gini measures unevenness, while Shannon entropy measures uncertainty. Their interpretation depends on distribution size and, for entropy, the number of nonzero citation categories.
- Interpretation: A journal citing or being cited only by itself has Gini equal to unity and Shannon entropy equal to zero, representing an extremely mono-disciplinary pattern.More distributed citation patterns are expected to have lower Gini and higher entropy.
- Gini coefficients: The Gini coefficient ranges from zero for an even distribution to (n – 1)/n for a completely uneven distribution.For large populations, the upper bound approaches one.
- Gini coefficients: Comparisons among smaller populations of varying size require normalizing Gini coefficients to a common maximum of one.The study introduces a normalized Gini coefficient for this purpose.
- Shannon entropy: Shannon entropy formalizes the uncertainty contained in a citation distribution.The paper applies it to citation distributions in both cited and citing directions.
- Shannon entropy: For 8,207 possible journals, the maximum information is log2(8,207) = 13.00 bits, so entropy does not require further normalization against that global maximum.Observed entropy is also assessed against a journal-specific maximum based on its nonzero cells.
Betweenness centrality
Betweenness centrality was evaluated across asymmetrical citation and symmetrical co-citation networks, including cosine-normalized representations. The procedure binarized matrices and compared cited and citing dimensions without imposing a cosine threshold.
- Network comparisons: Betweenness centrality was computed on both cited and citing network representations.The study compares asymmetrical citation matrices with symmetrical co-citation matrices derived from them.
- Normalization: Cosine normalization makes the matrix symmetrical, while normalization can be performed over either the cited or citing axis.These choices produce four evaluations for comparison.
- Definition: Betweenness centrality measures the proportion of geodesics between vertex pairs that include a given vertex.The matrix is first binarized to compute Freeman betweenness centrality.
- Thresholding: After binarization, betweenness in cosine-normalized vector space equals betweenness in the corresponding co-occurrence matrices when no cosine threshold is used.The study did not set a threshold because it would add a model parameter and complicate comparisons.
- Computation: Co-occurrence matrices offer a computational advantage over cosine-normalized matrices for the large dataset.The co-citation matrices are obtained by multiplying the asymmetrical citation matrix by its transpose.
Distance matrices (Rao-Stirling diversity)
The study compares Euclidean and (1 – cosine) distances for Rao-Stirling diversity, addressing size effects through normalization. Euclidean distances are intuitive but can treat equally distributed journals of different sizes as distant, whereas (1 – cosine) represents dissimilarity.
- Euclidean distances: Euclidean distances are familiar and intuitively accessible, but their interpretation depends on restrictive assumptions and geometry choices.They can also be transformed by scaling dimensions to represent different geometries.
- Size normalization: Identical citation distributions with different sizes can appear distant under Euclidean distance, so the data must be normalized for size.Using citation proportions instead of absolute frequencies neutralizes this size effect.
- Alternative normalization: Z-scores are also affected by citation-matrix zeros because they rely on averages and are influenced by the mean.This motivates considering cosine-based alternatives and size-normalized Euclidean distances.
- Cosine-based distances: Cosine is a similarity measure, whereas (1 – cosine) represents dissimilarity and can serve as a relevant distance measure.The cosine was proposed as a non-parametric similarity criterion because Pearson correlation is affected by many zeros in citation matrices.
- Rao-Stirling design: The analysis computes Rao-Stirling diversity across four combinations of normalized Euclidean or (1 – cosine) distances with cited or citing distributions.Relative frequency distributions provide the probability distribution needed for Rao-Stirling diversity.
Results
The results distinguish vector-based, network-based, and combined indicators, showing that each captures different aspects of journal interdisciplinarity. Shannon entropy and cosine-normalized betweenness yield interpretable results, while Rao-Stirling rankings depend strongly on the chosen distance measure and can be difficult to interpret.
- Vector-based indicators: Cited and citing rankings differ substantially: journals with diverse knowledge bases do not necessarily have diverse audiences.The cited and citing dimensions show rank-order correlations of ρ = –0.803 and ρ = –0.658, respectively.
- Vector-based indicators: Shannon entropy measures variety but is affected by journal size, whereas the Gini coefficient captures specificity or unevenness.Entropy therefore qualifies as a vector-based interdisciplinarity measure when correction for size effects is not the primary concern.
- Network indicators: Without normalization, Science, Nature, and PNAS rank prominently, but cosine normalization removes all three from the citing top-20 list.Normalization changes the apparent interdisciplinarity rankings and brings other journals, including information-science titles, to prominence.
- Network indicators: Betweenness centrality produces understandable results after normalization, with cosine-normalized network measures providing an interpretable indicator.The study reports a full listing of 8,207 journals across 12 indicators.
- Rao-Stirling diversity: Rao-Stirling diversity is heavily dependent on the distance measure: (1 – cosine) results are interpretable, whereas Euclidean-distance results are harder to interpret.The (1 – cosine)-based measure operates on average better than the Euclidean-based measure, although some rankings remain unconvincing.
- Rao-Stirling diversity: The indicator’s discriminating power can be weak, with differences sometimes appearing only in the third decimal.The paper also notes that varying exponents would complicate identification of a simple and robust indicator.
- Overall comparison: Overall, vector-based, matrix-based, and combined indicators provide insight into different aspects of interdisciplinarity rather than a single definitive measure.The paper concludes that Shannon entropy is useful at the vector level, while cosine-normalized betweenness and selected Rao-Stirling variants offer complementary information.
Relations among the various indicators
Factor analysis separates the indicators into dimensions associated with size, impact, and interdisciplinarity. Shannon entropy and cosine-based Rao-Stirling diversity align with interdisciplinarity, while betweenness remains more size-associated and Euclidean Rao-Stirling is difficult to interpret.
- 72.4% of the variance is explained by three factors, with the first associated with size and the third with impact.
- Entropy, the Gini coefficient, and Rao-Stirling diversity using (1 – cosine) constitute a dimension interpretable as interdisciplinarity.
- Betweenness centrality loads highest on the size factor even after normalization for size.
- Rao-Stirling diversity based on relative Euclidean distances loads negatively on the impact factor and differs from the other indicators.
- The cited and citing factor structures differ considerably, suggesting that data-matrix functionality matters more than correlations among indicators.
- Shannon entropy qualifies as a vector-based interdisciplinarity measure, whereas the Gini coefficient does not simply capture disciplinary specificity.
- Betweenness centrality and cosine-normalized Rao-Stirling diversity indicate different aspects of interdisciplinarity, but betweenness remains more associated with size.
Library and information science
A 61-journal library and information science analysis compares indicator structures after removing outliers and peripheral journals. Cosine-based Rao-Stirling diversity relates more consistently to entropy and is less correlated with journal size than betweenness or entropy.
- The analysis compares how these journals are cited by the 8,207-journal database and can also use a 61 x 61 citation matrix.
- The two distance measures are no longer correlated in the focused analysis, with ρ = 0.230 for (1 – cosine) and relative Euclidean distances.
- The factor structures become comparable between cited and citing dimensions and are considered reliable for the selected journals.
- 61 journals were analyzed after removing outliers such as Nature and Science and peripheral journals with few citations beyond self-citations.
- Entropy correlates with cosine-normalized betweenness at ρ = 0.830 and with cosine-based Rao-Stirling diversity at ρ = 0.732.
- Rao-Stirling diversity correlates less with total cites than betweenness centrality or Shannon entropy: ρ = 0.549 versus 0.880 and 0.793.
- Euclidean-distance Rao-Stirling diversity is not considered a good indicator, while other indicators rank the Journal of Informetrics between 48 and 51 of 61.
- The journal is ranked as disciplinary and differs in this respect from JASIST and Scientometrics.
Conclusions and discussion
The study finds that no single indicator captures every aspect of journal interdisciplinarity. Betweenness centrality in cosine-normalized matrices and Shannon entropy are useful, while Rao-Stirling results depend strongly on the distance measure.
- No robust and unambiguous indicator of interdisciplinarity was identified, supporting a multi-indicator approach.
- Three classes of indicators—network, vector-based, and diversity measures—were considered as candidates for journal interdisciplinarity.
- Shannon entropy captures both citation reach and citation spread, but correcting its size effect can elevate narrowly focused specialist journals.
- 16 journals reached 100% of the local maximum entropy, including Journal of Nanomaterials and Brain Cell Biology, despite citation coverage by six and two journals, respectively.
- Betweenness centrality based on cosine-normalized matrices qualifies as an interdisciplinarity indicator, with Science and Nature ranking 6th and 12th rather than at the top.
- Rao-Stirling diversity was highly sensitive to distance choice: relative Euclidean results were difficult to interpret, while (1 – cosine) distances were debatable.