Gender differences in grant peer review: A meta-analysis

Introduction

• The paper asks whether applicant gender affects grant peer-review decisions, because gender is functionally irrelevant to scientific merit and could compromise objectivity if it shapes decisions.The authors frame this as a question about an overall gender effect across studies.Source: Introduction · pp. 3, 5
• Earlier narrative reviews described little evidence of gender bias in grant awards, but did not quantitatively synthesize study results.The cited reviews reported mixed findings, including a Swedish study indicating bias and other studies in similar settings finding none.Source: Introduction · pp. 3–4
• Meta-analysis offers an overall estimate across studies, but assumes sufficient similarity in study properties such as methods and sampling.The paper notes that studies of grant peer review vary in these properties, although many report comparable applicant and approval proportions by gender.Source: Introduction · p. 4
• Because grant peer review is studied in natural settings, higher approval rates cannot be unambiguously attributed to bias rather than applicants’ scientific merit.The authors therefore restrict their analysis to non-experimental studies, where causal demonstration is difficult.Source: Introduction · p. 5

Locating studies

• The literature search combined prior review references and journal contents with searches of publication and grey-literature sources, database keywords, citing publications, and author inquiries.Keywords for computerized searches were developed from a bibliogram of initially located studies.Source: Locating studies · pp. 5–6
• The search was constrained by peer review’s confidentiality, unpublished findings, and documents available only in their native language.The authors report that these conditions can make studies difficult to locate or their findings difficult to access.Source: Locating studies · pp. 6–7

Studies for the meta-analysis

• The meta-analysis included gender-proportion data from 21 studies covering 66 peer-review procedures, with about three procedures represented per study.Each procedure was characterized by the proportions of women and men among applicants and among approved applicants.Source: Studies for the meta-analysis · p. 7
• The 66 procedures comprised 40 research-application reviews and 26 postgraduate-fellowship reviews.The procedures covered funding institutions in Australia, North America, and Europe, alongside multiple disciplinary groupings.Source: Studies for the meta-analysis · p. 8

Statistical methods

• The outcome was the odds of approval among women applicants divided by the odds among men applicants, estimated from the reported gender proportions.The model analyzed log odds ratios because the effect-size estimates violated the normality assumption.Source: Statistical methods · pp. 8–9
• A generalized linear mixed model estimated the overall gender effect while allowing it to vary between peer-review procedures and between studies.Its random components represent variation in regression effects at both levels; the data are structured as applicants within procedures within studies.Source: Statistical methods · pp. 8–9

Results

• Men had statistically significantly greater odds of grant approval than women by about 7% across the procedures analyzed.The estimate summarizes 66 peer-review procedures from 21 studies.Source: Results · p. 10, §1.07 · p. 11

1.07. In other words, in grant peer review procedures, men have on average

• The authors characterize the overall gender effect as a robust average based on 353,725 applications.The model’s fixed part estimates the overall gender regression, while its random part represents variation across procedures and studies.Source: Results · p. 10, §1.07 · p. 11
• The model showed statistically significant variability in the gender effect both between peer-review procedures and between studies.The estimated variances were 0.04 for procedures and 0.07 for studies.Source: §1.07 · p. 11
• Individual procedure estimates ranged from a mean log odds ratio of -0.26 to 0.20, with only a few procedures favoring women.Figure 1 presents Empirical Bayes estimates and credible intervals for individual procedures.Source: §1.07 · pp. 11–12

• Men applicants had about 7% greater odds of grant approval than women (about 15:14), despite substantial variation across procedures, contrary to narrative reviews.Source: Discussion · p. 12
• In a hypothetical 100,000 applications split evenly by gender with half approved, expected rates were 52% for men and 48% for women (26,000 vs. 24,000), a 2,000-approval gap.Source: Discussion · p. 12
• The cause remains unknown: models including procedural characteristics such as discipline or application cohort did not converge, perhaps because they became too complex or those characteristics had no effect.The studies covered periods from 1979 to 2004, during which the authors assume progress was made in reducing gender bias in science and funding.Source: Discussion · p. 13
• Gender differences in grant peer review were not due to differences in scientific productivity before grant application; suggested explanations include fewer women applicants, bias, and fewer women in senior decision-making positions.Valien (1999) suggested that adequate numbers of women and men in an evaluated group could make sex less distinctive.Source: Discussion · pp. 13–14
• Masking applicants’ gender is one proposed remedy, but it is essentially impossible for short communications that require personal knowledge of the author (Daniel, 1993, 2004; Ziman, 1968).Source: Discussion · p. 14
• At the NIH Pioneer Award, no women were among nine winners in 2004, while about half of 13 winners were women in 2005; cited measures included encouraging women to apply, accepting only self-nominations, and reviewer training (Mervis, 2004; Mervis, 2005).Source: Discussion · p. 15
• Table 3 identifies the studies included in the meta-analysis and provides their references.Source: Table 3 · p. 23
• The meta-analysis estimates an applicant-gender effect on grant approval and models variability between procedures and studies.The fixed part estimates the gender effect; the random part represents variability between procedures and studies and heterogeneity of effects.Source: Table 4 · p. 25, Discussion · p. 25
• Figure 1 presents mean log-odds ratios and credible intervals for each peer review procedure, with negative values indicating preference for men and positive values preference for women.Source: Figure 1 · p. 26