Source-linked AI summary
Interdisciplinarity as Diversity in Citation Patterns among Journals: Rao-Stirling Diversity, Relative Variety, and the Gini coefficient
Loet Leydesdorff, Caroline S. Wagner, Lutz Bornmann
TL;DR
The paper addresses unresolved definitions and anomalous Rao-Stirling results in measuring interdisciplinarity. It proposes DIV, which measures variety, balance, and disparity independently before combining them, and applies it to journal citation data. The authors report less puzzling empirical results and stronger correlation with betweenness centrality than RS, while noting that indicator choice lacks an obvious ground truth.
Problem
Measuring interdisciplinarity remains unsettled because Rao-Stirling diversity can produce anomalous outcomes and the components of diversity are difficult to combine without losing information or validity.
Method
DIV independently operationalizes variety, balance, and disparity, uses the Gini coefficient for balance, and then combines the components ex post.
Results
DIV produces less puzzling empirical results and correlates significantly more with betweenness centrality than RS diversity.
Takeaways & Limitations
Diversity components can be measured independently and combined to provide a more informed interdisciplinarity measure than the dual-concept treatment associated with RS.
Takeaways & Limitations
There is no ground truth for interdisciplinarity, so the study identifies no obvious criterion for choosing one indicator over another; RS also does not cover balance despite its importance.
Abstract
from arXiv · showhide
Questions of definition and measurement continue to constrain a consensus on the measurement of interdisciplinarity. Using Rao-Stirling (RS) Diversity produces sometimes anomalous results. We argue that these unexpected outcomes can be related to the use of "dual-concept diversity" which combines "variety" and "balance" in the definitions (ex ante). We propose to modify RS Diversity into a new indicator (DIV) which operationalizes variety, balance, and disparity independently and then combines them ex post. "Balance" can be measured using the Gini coefficient. We apply DIV to the aggregated citation patterns of 11,487 journals covered by the Journal Citation Reports 2016 of the Science Citation Index and the Social Sciences Citation Index as an empirical domain and, in more detail, to the citation patterns of 85 journals assigned to the Web-of-Science category "information science & library science" in both the cited and citing directions. We compare the results of the indicators and show that DIV provides improved results in terms of distinguishing between interdisciplinary knowledge integration (citing) versus knowledge diffusion (cited). The new diversity indicator and RS diversity measure different features. A routine for the measurement of the various operationalizations of diversity (in any data matrix) is made available online.
1. Introduction
The paper revisits how interdisciplinarity is measured as diversity, addressing anomalous Rao-Stirling results by independently operationalizing variety, balance, and disparity before combining them. It compares these indicators across journal citation data and a focused information-science case study.
- Interdisciplinarity is increasingly defined through three diversity features: variety, balance, and disparity.
- The central problem is how to combine variety, balance, and disparity without losing information or validity.
- Rao-Stirling Diversity can yield counterintuitive comparisons, including Rotterdam and Jerusalem ranking above Shanghai and Paris.
- The proposed measure reverses that city ranking, placing Shanghai first and Rotterdam 16th among twenty cities.
- The paper separates variety, balance, and disparity operationally, using the Gini coefficient for balance, then compares the resulting measures with RS diversity.
- The empirical analysis uses 11,487 journals from the 2016 Science Citation Index and Social Sciences Citation Index, plus 85 information-science and library-science journals.
- The study asks whether the indicators measure the same or different dimensions of interdisciplinarity and supplies routines for computing them on data matrices.
2. The Measurement of Interdisciplinarity in terms of Diversity
The section frames interdisciplinarity as diversity composed of variety, balance, and disparity, and examines how these components are operationalized in competing indicators. It identifies a problem with combining variety and balance ex ante in dual-concept measures such as Rao-Stirling diversity.
- Stirling’s framework defines diversity through three components: variety, balance, and disparity.
- Each diversity component is necessary but insufficient, and diversity should increase when any one rises while the other two remain fixed.This is the monotonicity requirement.
- Earlier interdisciplinarity measures often captured only variety, or combined variety with disparity using distributions and distances among categories.Portfolio spreads across Web-of-Science categories operationalize variety, while map distances provide a measure of disparity.
- Rao-Stirling diversity combines disparity with the Simpson Index, thereby inheriting the Simpson Index’s dual treatment of variety and balance.The resulting indicator multiplies a disparity term by a Simpson-based term rather than measuring all three components independently.
- This dual-concept formulation makes the relative weighting of variety and balance a central measurement problem when systems differ across these dimensions.The concern is especially relevant when no system is unequivocally more diverse than another on both dimensions.
- The section motivates revisiting the three components so they can be operationalized separately rather than combined ex ante.
3. Data and Methods
The study tests diversity measures on aggregated citation data from the 2016 Journal Citation Reports and examines information science and library science journals in cited and citing directions. Its design compares measures across a broad journal network and a focused 85-journal subset.
- 3.1. Data: The empirical analysis uses aggregated citation relations among 11,487 journals from the 2016 Science Citation Index and Social Sciences Citation Index.
- 3.1. Data: A focused analysis examines 85 journals in the Web of Science category “information science and library science.”The study uses analogous cited and citing distributions for this subset.
- 3.1. Data: The analysis distinguishes citing distributions as knowledge integration and cited distributions as knowledge diffusion.
- 3.1. Data: In the information science and library science subset, JASIST ranks third for citing diversity while Scientometrics ranks 45th; in the cited dimension, Scientometrics ranks 70th among 86.The cited-direction ranking is described as counter-intuitive in the paper.
- 3.1. Data: The routine compares RS diversity, DIV, Gini, Simpson, Shannon, disparity, and relative and absolute variety on vectorized citation data.It operates on column vectors and can analyze the opposite direction after matrix transposition.
Transform> 1-Mode to 2-Mode.
The routine transforms citation data into comparable class vectors, computes pairwise dissimilarities, and outputs multiple diversity components and indicators.
- Matrix transformation: Citation co-occurrences are converted into a one-mode matrix whose vectors are compared across classes.The resulting dissimilarity matrix supports subsequent diversity calculations.
- Distance measure: The study uses 1 − cosine as its disparity distance because it is non-parametric, ranges from 0 to 1, and disregards zeros.Other measures, including Jaccard and Euclidean distance, can also be used.
- Balance measure: The Gini coefficient is computed from observed values using either pairwise differences or ranked values and their ranks.The passages define x, n, the mean, and rank i for these formulations.
- Outputs: The output includes RS diversity, true diversity, DIV, Gini, Simpson, Shannon, disparity, and absolute and relative variety.The routine processes the lower triangle of the symmetric dissimilarity matrix and doubles the result when appropriate.
- Comparison indicators: Betweenness centrality and journal impact factors are added separately for comparison.
4. Results
The results show that diversity indicators rank journals differently across citing and cited patterns, with DIV more closely associated with interdisciplinary structure than RS diversity.
- Interpretive qualification: Some citing-direction rankings may reflect local knowledge integration rather than the form of interdisciplinarity valued in science policy.The passages specifically qualify this interpretation for journals from less-developed nations.
- LIS journals: DIV identifies more obvious interdisciplinary candidates in the cited direction than RS diversity, including journals that RS ranks much lower.Scientometrics leads the DIV cited ranking but is 69th among 85 journals when ranked using RS.
- Full journal set: Cited-direction rankings place broad-impact journals such as PLOS ONE, Scientific Reports, and major medical journals among highly interdisciplinary outlets.The cited direction also brings general social science journals to the fore, while Gini and Simpson rankings emphasize different journal groups.
- Full journal set: Science and Nature are absent from the citing-direction top 25 because their referencing is described as precise and disciplined.Their broad citation by other journals is associated with their status, distinguishing citing integration from cited diffusion.
- Full journal set: DIV correlates more strongly with betweenness centrality than RS diversity in both citing and cited dimensions.For the full set, the reported correlations are ρ = .51 versus .10 in citing and .66 versus .41 in cited patterns.
- Full journal set: DIV and RS diversity produce different rankings and correlations, indicating that they measure different features of citation patterns.Their rank-order correlations are .19 in citing patterns and .35 in cited patterns.
- Component correlations: RS correlates most strongly with disparity, whereas DIV correlates more with Gini, Simpson, Shannon, and relative variety.The authors attribute this difference to RS multiplying disparity by only the Simpson index, while DIV multiplies three components.
- LIS journals: For the 85 LIS journals, DIV correlates with BC at ρ = .59 in citing patterns, versus .35 for RS diversity.Both indicators show no significant correlation with JIF2 in either citing or cited directions.
5. Factor Analysis
Factor analysis separates the indicators into two dimensions: DIV aligns with interdisciplinary diffusion, while RS aligns more strongly with interdisciplinary knowledge integration.
- Method: The analysis uses principal component extraction with varimax rotation and focuses on factor loadings above 0.5.Gini and Simpson are treated as components of the diversity measures and therefore excluded as redundant variables.
- Factor structure: Two factors have eigenvalues above 1 and together explain 65.1% of the variance.
- Factor interpretation: DIV in the cited direction has the highest loading on the first factor and is completely uncoupled from Factor 2.This factor is interpreted alongside interdisciplinary diffusion.
- Factor interpretation: Factor 2 couples to interdisciplinary knowledge integration, with the highest loading for RS in the citing direction.
6. Range
Across 11,487 journals, RS diversity assigns high diversity values to most journals, whereas DIV spans a larger range that permits more refined measurement.
- Observed ranges: 10,264 of 11,487 journals, or 89.3%, have RS-diversity values above 0.5.Thus, most journals are indicated as diverse by RS diversity.
- Interpretation: The larger range of DIV allows more refined measurement.
7. Summary and conclusions
The study proposes DIV as an alternative to Rao-Stirling diversity, independently operationalizing variety, balance, and disparity before combining them. It reports that DIV addresses anomalous and counterintuitive RS results while distinguishing the components more specifically, although empirical results alone cannot establish a ground truth for interdisciplinarity.
- Contribution: DIV independently operationalizes variety, balance, and disparity, then combines the three components ex post.This replaces RS’s ex ante combination of variety and balance as a dual-concept indicator.
- Operationalization: Relative variety is the number of classes in use divided by the number of classes available, while disparity uses a distance measure; this study uses (1 – cosine).Balance is operationalized with the Gini coefficient, and the components are bounded between zero and one.
- Properties: DIV is monotonic: diversity increases in each component when the other two remain unchanged.The indicator multiplies the three component values, producing a result bounded between zero and one.
- Empirical findings: DIV produces less puzzling and counterintuitive empirical results than RS and correlates significantly more with betweenness centrality.The authors present these as advantages of the new indicator in comparison with RS diversity.
- Limitations: Because interdisciplinarity has no ground truth, different indicators cannot be selected from empirical results alone unless those results show obvious invalidity.The authors also note that disciplines are social constructs with fluid boundaries, limiting straightforward indicator evaluation.
- Motivation: The study’s motivation is that RS measurements of interdisciplinarity often produced incomprehensible or counterintuitive results linked to combining variety, balance, and disparity ex ante.The authors identify the dual-concept treatment of variety and balance as the central problem and argue that it lacks theoretical and practical justification.
Annex I
Annex I documents the software workflow for computing diversity indicators from Pajek two-mode matrices and reports correlation tables for broad journal coverage and 85 LIS journals.
- Software workflow: mode2div.exe computes diversity measures along the column vectors of a Pajek .net two-mode matrix.The program and data file must be stored in the same folder, and the user is prompted for the data filename.
- Software workflow: 1-mode matrices must first be transformed into two-mode matrices before being saved for analysis.Pajek provides the required transformation through Network > Create New Network > Transform > 1-Mode to 2-Mode.
- Software workflow: A cooccurrence network is generated from the two-mode matrix by projecting its rows into a one-mode network.The resulting cooccurrences are taken along rows when column vectors are compared.
- Indicators and correlation tables: RS and DIV measure diversity at the matrix level, including disparity, whereas Gini and Simpson measure diversity at the vector level.The annex also provides correlation tables for DIV, RS diversity, Betweenness Centrality, and Journal Impact Factor across 11,487 journals and for DIV, RS diversity, and Betweenness Centrality across 85 LIS journals.
- Software workflow: The output file div_col.dbf contains diversity indicators for all units of analysis along the column dimension.Previous versions of the output file are overwritten during processing.
- Indicators and correlation tables: The annex includes correlation tables for citing and cited journal vectors, using Spearman’s ρ in the upper triangle and Pearson’s r in the lower triangle.Separate tables cover the full JCR 2016 journal set and the 85-journal LIS subset.