PaperlayerView exact revision on arXiv

Source-linked AI summary

Prominent but Less Productive: The Impact of Interdisciplinarity on Scientists' Research

Erin Leahey, Christine Beckman, Taryn Stanko

arXiv:1510.06802v1cs.DL

TL;DR

Interdisciplinary research is promoted for potentially transformative science, but its production-side costs remain less examined. An analysis of 854 PhD-level scientists finds that production penalties outweigh reception benefits, hampering overall productivity.

  • Problem

    The paper examines whether interdisciplinary research carries production-side costs, including cognitive demands and slower publication progress.

  • Method

    The study analyzes 854 PhD-level scientists and measures interdisciplinary research through field variety and distance.

  • Results

    The standardized production direct effect = -0.06 outweighed the reception benefit of +0.04, while overall productivity was hampered.

  • Takeaways & Limitations

    Interdisciplinary research can incur a clear production penalty, with lower overall productivity despite a reception benefit.

  • Takeaways & Limitations

    The sample’s older age and unusually high productivity prompted author-led subset analyses.

Abstract

from arXiv · show

Inter-disciplinary research (IDR) is being promoted by federal agencies and universities nationwide because it presumably spurs transformative, innovative science. In this paper we bring empirical data to assess whether IDR is indeed beneficial, and whether costs accompany potential benefits.

PRODUCTION PENALTIES OF INTERDISCIPLINARY RESEARCH

Interdisciplinary research may incur production penalties because cognitive, technical, logistical, collaborative, and review challenges make research slower and harder to produce. Interview evidence suggests these burdens reduce productivity, motivating the hypothesis that IDR is associated with lower productivity.

  • Cognitive and mastery challenges: IDR may make mastery harder by dispersing finite cognitive resources across multiple categories.The paper focuses on technical and logistical hurdles that arise when spanning domains.
  • Collaborative challenges: Interdisciplinary collaboration can generate epistemological or methodological conflict, increasing coordination costs, tension, and role strain.These divergences may outweigh collaboration’s general benefits and slow progress toward publication.
  • Review-stage challenges: Interdisciplinary work may face slower and more difficult review because experts from different fields disagree about papers’ merits and evaluate them differently.The paper links these review challenges to lower publication rates.
  • Interview evidence: Interview evidence from successful interdisciplinary scholars indicates that IDR requires additional time, commitment, and effort and is mentally more taxing.Interviewees also described taking longer from conception to publication and experiencing reduced productivity.
  • Hypothesis: H1: IDR is associated with lower productivity.The authors propose that scientists more engaged with interdisciplinary work publish fewer research articles than scientists less engaged in IDR.

RECEPTION BENEFITS OF INTERDISCIPLINARY RESEARCH

The section argues that interdisciplinary research should gain visibility despite potential reception penalties, making IDR a high-risk, high-reward activity. It further proposes that citation variability and the distance between fields shape these reception effects.

  • Reception benefits: H2 predicts that IDR is associated with higher visibility, measured through citation counts even after controlling for prospective audience size.The paper treats citations as an indicator of scholarly visibility and reception.
  • Reception benefits: H3 predicts that IDR is associated with greater variability in visibility, combining the expectation of average prominence with a higher risk of being ignored.The authors characterize IDR as high-risk, high-reward: it may produce breakthroughs or failures.
  • Distance and interdisciplinarity: The section distinguishes spanning categories from spanning distant categories, arguing that cognitive distance increases both category-spanning challenges and its potential utility and value.Examples contrast cognitively similar fields, such as civil and chemical engineering, with distant fields, such as geography and optics.
  • Distance and interdisciplinarity: H4 predicts that an IDR measure excluding distance will have weaker effects on productivity and visibility.The hypothesis treats distance between spanned fields as consequential for both outcomes.

INTELLECTUAL CONTEXT and IDR

The paper argues that IDR’s benefits and penalties depend on scientists’ intellectual context, with highly interdisciplinary fields expected to amplify visibility gains and reduce productivity costs. It assesses whether IDR increases citations while reducing productivity, and whether field-level expectations moderate these effects.

  • INTELLECTUAL CONTEXT and IDR: Life sciences are more interdisciplinary than engineering and math, with 36% of dissertations self-classified as interdisciplinary versus 26% and 21%, respectively.Using a cognitive-dissimilarity measure, Biotechnology (mean=0.654) and Medicine (mean=0.664) also exceed Electrical Engineering (mean=0.53).
  • INTELLECTUAL CONTEXT and IDR: The paper expects IDR penalties to arise mainly during production through cognitive and communicative challenges and reduced productivity.Less interdisciplinary fields may lack models, training, and amenable review processes for producing IDR.
  • INTELLECTUAL CONTEXT and IDR: Highly interdisciplinary fields are expected to provide stronger visibility benefits and weaker productivity penalties for IDR than less interdisciplinary fields.The paper formulates these expectations as H5a and H5b.
  • INTELLECTUAL CONTEXT and IDR: The paper expects IDR benefits to emerge during reception through increased citations, when scholarship is evaluated in relation to otherwise similar work.This empirical focus is intended to ground claims about the value of IDR.

Sample

The study analyzes archival publication data from scientists affiliated with 52 NSF-funded industry/university-cooperative research centers, focusing on 854 PhD-level scientists with publication records. It combines approximately 32,000 Web of Science articles and cited references with additional field-, university-, and individual-level data, supplemented by survey and journal-timing information.

  • Sample: The scientists were associated with 52 NSF-funded industry/university-cooperative research centers housed at universities and partly supported by industry.The centers covered diverse research areas, including biosurface physics, civil and environmental engineering, and architectural science.
  • Sample: The sample comprises 854 PhD-level scientists with publication records as of 2005, drawn from IUCRC members identified in 2003.The broader IUCRC membership included doctoral students, post-docs, and faculty at all career stages.
  • Data: Approximately 32,000 articles published by center affiliates were collected from Thomson Reuters Web of Science, including authorship, institutional representation, journals, and citations.The study also collected all articles referenced by these focal articles and their Web of Science subject categories.
  • Data: Web of Science subject categories were used to measure each scholar’s interdisciplinarity, alongside field-, university-, and individual-level data.Supplementary analyses used a survey of center-based scientists, publicly available journal turn-around times, and unpublished working papers for a subset of authors.

Dependent Variables

The study measures productivity by scholars’ WoS article counts through 2005 and visibility by forward citations accrued to each WoS-indexed article through 2010. Analyses use person- and person-year-level measures, with transformations or count models addressing skewness and citation variability.

  • Productivity: Productivity is measured as the total number of WoS-journal articles published from each scholar’s first publication year through 2005.This conservative measure excludes book chapters and articles in less internationally recognized journals.
  • Productivity: Journal impact-factor-weighted productivity is excluded because it confounds quantity with quality, which IDR may affect differently.The study instead treats productivity and visibility as distinct outcomes.
  • Productivity: The analyses examine article counts at the person and person-year levels.Article counts are standard in scientific-productivity research, remain correlated with co-author-weighted counts, and are assessed for robustness using that alternative measure.
  • Visibility: Visibility is measured using forward citations accrued to each WoS-indexed article as of 2010.Paper-level citations are treated as more precise measures of impact than journal prestige, while journal impact factor is controlled in all models.
  • Visibility: Person- and person-year-level visibility measures are obtained by averaging article citations, while citation variability is measured with scientists’ citation standard deviation.Because aggregated productivity and visibility are highly right-skewed, analyses use natural logs or negative-binomial count models.

Independent Variables

The study measures interdisciplinarity through a paper-level IDR index based on the variety, balance, and cognitive distance of subject categories in references. It aggregates IDR to person-year and field levels, excludes zero-IDR observations, and validates the measure against related indicators and disciplinary fields.

  • Interdisciplinary Research (IDR): IDR scores are averaged across each scientist’s papers for person-level and person-year analyses, with person-year scores based on papers published that year.The measure pools subject categories from each focal paper’s references rather than from the focal paper itself or papers citing it.
  • Interdisciplinary Research (IDR): Paper-level IDR integrates the variety, evenness, and cognitive distance of Web of Science subject categories assigned to references.The index ranges from 0 to 1, with higher scores indicating greater interdisciplinarity.
  • Interdisciplinary Research (IDR): 551 papers and 99 person-years have zero IDR scores, so analyses exclude zero-IDR observations because these cases reflect very low productivity years.Including zero-IDR observations leaves visibility results robust but makes productivity effects curvilinear because both zero and high IDR scores are associated with low productivity.
  • Interdisciplinary Research (IDR): The IDR measure shows convergent validity, correlating positively with focal-paper subject-category counts (r=0.18) and averaging 0.71, 0.71, and 0.70 in Science, Nature, and PNAS versus 0.62 in the sample.The authors also report that the score increases when scientists reference more and more unrelated fields.

Control Variables

The analyses control for person-, journal-, field-, and paper-level factors that could influence interdisciplinarity, productivity, or visibility. Center-level controls are omitted because none were statistically significant and clustering was minimal.

  • Person-level controls: Person-level models control for gender, professional age, current-institution status, and PhD-granting institution quality.Gender was inferred from first names, pictures, and pronouns; professional age was calculated from PhD receipt year to 2005; status used National Academy of Sciences faculty counts; PhD-institution rankings were reverse-coded so higher values indicated higher quality.
  • Journal- and field-level controls: Journal- and field-level controls include mean journal impact factor, field-specific journal turn-around-time, average citations per paper, and field size.Field size was measured as the number of PhDs produced in a recent year to address audience-size explanations for visibility.
  • Journal- and field-level controls: The models also control for each paper’s potential reach using the number of SCs classifying the focal paper itself.The authors presume that papers classified in multiple fields reach a larger body of scholars, potentially boosting citations.
  • Center-level controls: Center-level controls were excluded because none reached statistical significance and the intra-class correlation coefficient was 0.11, indicating minimal clustering by center.This decision applied despite the sample being center-based.
  • Paper-level and person-year controls: Paper-level and person-year analyses control for team composition, prior collaboration, lagged publication experience, lagged citations, and publication year.Team composition includes author count and whether the authors had published together before; lagged citations were logged to remedy skewness, and publication year accounts for older papers having more time to accrue citations.

Statistical Approach

The study uses analyses at paper, person-year, and person levels to match outcomes to appropriate units and test robustness. Person-level path models examine productivity, visibility, and their direct and indirect relationships, while fixed-effects panel models address within-person variation and causal direction.

  • Multi-level design: Analyses span paper, person-year, and person levels to model outcomes appropriately, incorporate level-specific controls, and assess robustness.Productivity cannot be measured at the paper level, motivating person-year models, while visibility is modeled at both paper and person-year levels.
  • Person-level path analysis: n=854 scientists; person-level path analysis simultaneously models productivity, visibility, their inter-relationship, and direct and indirect effects.The models can represent pathways such as IDR→productivity→visibility and explicitly incorporate intervening variables.
  • Missing-data handling: Full information maximum likelihood includes all available data and avoids listwise deletion and data imputation in the path models.The Stata 13 sem package is used to estimate path models and handle missing data.
  • Robustness checks: Multiple imputation is used for some missing variables, but results are the same without multiple imputation.Field-level IDR is tested with random-effects visibility models and population-averaged negative binomial productivity models using robust standard errors.

RESULTS

IDR is associated with lower scholarly productivity but greater visibility and citation variability. The productivity penalty is strongest when IDR spans cognitively dissimilar fields and is weaker in highly interdisciplinary fields.

  • Visibility: +0.69*** indicates that a 10% increase in IDR increases a scholar’s average citations by 6.9%, including with person and year fixed-effects.The visibility benefit remains after excluding group-authored papers and omitting 171 papers from multidisciplinary, high-impact journals.
  • Cognitive distance: Cognitive dissimilarity contributes to IDR’s visibility benefit and production penalty, whereas drawing on related fields does not hinder productivity and slightly improves citations.The results suggest that spanning unrelated fields is more difficult to produce and publish, consistent with IDR being cognitively difficult and slow when blending disparate fields.
  • Field context: Field-level IDR has positive and significant effects on productivity and visibility, weakening IDR’s productivity penalty and strengthening its citation benefit.The field-specific analyses find a weaker productivity effect in life sciences than electrical engineering, where IDR is not statistically significant; high-IDR fields may impose no penalty.
  • Controls and correlates: The main findings persist with controls; women are not more likely than men to engage in IDR, while university status has an inverted U-shaped relationship with IDR.Scientists at low- and high-status universities engage in IDR, whereas those at middle-status universities occupy a precarious position.

Drivers of the Productivity Penalty

The productivity penalty associated with interdisciplinary research appears to arise primarily from cognitive and collaborative challenges during research planning, coordination, and execution. Evidence provides little support for review-stage penalties, while repeated collaboration reduces the penalty and multidisciplinarity is associated with higher productivity.

  • Cognitive and Collaborative Challenges: Repeatedly working with a similar set of collaborators reduces the productivity penalty (b=1.51*, se=.077).This suggests that collaborators may learn to work together better over time.
  • Cognitive and Collaborative Challenges: Scientists conducting multidisciplinary research are more productive (b=0.58*, SE=0.31), whereas only scientists publishing interdisciplinary research are less productive.Multidisciplinarity requires diverse expertise and networks but not integration or coordination of diverse fields within a single paper.

Reskin 2003).

Robustness checks indicate that center affiliation and sample composition do not explain the reported positive visibility and negative productivity effects of IDR. Additional analyses also suggest that these effects are not driven by inherent paper quality differences.

  • Sample selection concerns: Fixed-effects models using papers before and after center founding retain significant positive visibility and negative productivity effects of IDR.Restricting the sample to papers published five years before and after center founding does not eliminate the main effects.
  • Sample selection concerns: Center affiliation does not significantly strengthen IDR’s visibility benefit or reduce its productivity cost.The interaction between IDR and center affiliation is not significant for either outcome.
  • Sample selection concerns: Compared with broader scientist and paper populations, the sample is older, more male, and more productive but similar in conventionality and slightly less novel.These comparisons suggest center-based scientists are not exceptional innovative outliers, although the design cannot completely rule out distinctive characteristics.
  • Sample selection concerns: The results hold for younger scientists and both men and women, while the IDR penalty is stronger for older scientists but has a very small coefficient.The IDR-by-age interaction is b= -0.012***, and female-by-IDR interactions are not statistically significant.
  • Quality of IDR papers: IDR’s citation benefit persists after controlling for journal impact, journal fixed effects, and low-impact journals, while review turnaround times do not differ by journal IDR.These checks, alongside controls for individual, journal, and field quality, support attributing increased visibility and reduced productivity to IDR rather than unobserved heterogeneity.

DISCUSSION

Interdisciplinary research increases scientists’ visibility and citations but reduces publication productivity, with the productivity penalty outweighing the direct citation benefit. The effects reflect cognitive and coordination costs, vary across fields and types of interdisciplinarity, and imply that scholarship evaluation should account for both individual costs and societal benefits.

  • Reception: IDR provides greater visibility and more overall citations but is high-risk, high-reward work with higher variation in citations.High-IDR papers are more highly cited, and interdisciplinary scientists’ papers are likely to be cited by other interdisciplinary papers.
  • Core findings: -0.06 versus +0.04: IDR’s standardized productivity penalty outweighs its standardized direct citation benefit.Compared with a scholar at the 20th percentile of IDR, one at the 80th percentile produces 24 rather than 26 articles and garners 297 rather than 253 citations.
  • Mechanisms: IDR’s productivity penalty appears to arise from communication and coordination hurdles in the research process, rather than from peer review.The authors describe interdisciplinary work as requiring more time, effort, diligence, and coordination to master disparate fields and collaborate across disciplines.
  • Field differences: The production penalty is more severe in low-IDR fields such as electrical engineering than in high-IDR fields such as life sciences.The authors suggest that training and IDR-compatible peer review may reduce cognitive and collaborative challenges.
  • Types of interdisciplinarity: Cognitive distance across disparate fields drives both IDR’s citation benefit and its publication penalty, whereas spanning similar or dissimilar subfields preserves visibility without the penalty.Papers spanning similar and dissimilar subfields receive more citations, but the productivity penalty does not hold for that distinction.
  • Implications: Although IDR imposes substantial individual-level costs, its societal benefits in producing more useful and valuable science appear clear, requiring reassessment of scholarship evaluation.The authors note that more research is needed on individual scientists’ career outcomes and on larger samples of subsequent publication.

Appendix A1. Construction of the Interdisciplinary Research (IDR) Measure for 3 Hypothetical Articles · Appendix A3. Robustness Checks: Regressions of Productivity and Visibility on IDRa

Appendix A1 defines IDR with Porter’s Integration measure, combining disciplinary variety, balance, and cognitive similarity across Web of Science Subject Categories, while Appendix A3 reports fixed-effects robustness regressions linking IDR to visibility and productivity. The hypothetical examples show that distant disciplinary combinations produce higher IDR than closely related combinations, and the regressions estimate positive visibility and negative productivity associations for IDR.

  • Appendix A1. Construction of the Interdisciplinary Research (IDR) Measure for 3 Hypothetical Articles: Porter’s Integration measure captures IDR through disciplinary variety, balance, and cognitive similarity combined into one index.Disciplines are proxied by 244 Web of Science Subject Categories; the hypothetical example assumes four categories.
  • Appendix A1. Construction of the Interdisciplinary Research (IDR) Measure for 3 Hypothetical Articles: Article 1 references two highly related categories, with Sij=0.35, and therefore receives a low IDR score.The passage identifies SC2 and SC3 as the highly related categories.
  • Appendix A1. Construction of the Interdisciplinary Research (IDR) Measure for 3 Hypothetical Articles: Article 2 references all four categories, including distant SC2 and SC4 with Sij=0.0001, and therefore receives a high IDR score.Its greater disciplinary variety and inclusion of distant categories distinguish the example from the other articles.
  • Appendix A1. Construction of the Interdisciplinary Research (IDR) Measure for 3 Hypothetical Articles: Article 3 references two distant categories, SC3 and SC4 with Sij=0.0011, so its IDR score exceeds Article 2’s.The example demonstrates that cognitive distance affects IDR even when the number of referenced categories is limited.
  • Appendix A1. Construction of the Interdisciplinary Research (IDR) Measure for 3 Hypothetical Articles: The three examples’ weighted similarity terms are 0.3146, 0.7361, and 0.6802, respectively, while their complementary IDR values are 0.6854, 0.2639, and 0.3198.The reported complement is calculated as 1 - ∑ij Sij * Pi * Pj.
  • Appendix A3. Robustness Checks: Regressions of Productivity and Visibility on IDRa: 0.276** is the IDR coefficient for visibility in Model 1, -0.312* is the IDR coefficient for productivity in Model 2, and 0.485*** is the IDR coefficient for visibility in Model 3.The appendix labels all three specifications as fixed-effects models.
  • Appendix A3. Robustness Checks: Regressions of Productivity and Visibility on IDRa: The regressions model visibility at the paper level and productivity at the person-year level, using person fixed-effects and publications from five years before and after center formation.The reported significance thresholds are *** p<.01, ** p<.05, and * p<.10 in two-tailed tests.
Loading 1510.06802v1…