Source-linked AI summary
Mandated data archiving greatly improves access to research data
Timothy H. Vines, Rose L. Andrew, Dan G. Bock, Michelle T. Franklin, Kimberly J. Gilbert, Nolan C. Kane, Jean-Sébastien Moore, Brook T. Moyers, Sébastien Renaut, Diana J. Rennison, Thor Veen, Sam Yeaman
TL;DR
Research data may be difficult to access, so this paper compares journal archiving policies and author requests; mandatory archiving with accessibility statements most strongly improves online data availability.
Problem
The study evaluates whether obtaining research data directly from authors is an effective approach to improving data access.
Method
The study compares journal-based mandatory archiving policies and requests data directly from authors.
Results
974 times higher odds of finding data online were associated with a mandatory policy plus an accessibility statement compared with no policy.
Takeaways & Limitations
Journal-based data archiving policies can be very effective for improving research-data accessibility.
Takeaways & Limitations
Requesting data by email upset some authors.
Abstract
from arXiv · showhide
The data underlying scientific papers should be accessible to researchers both now and in the future, but how best can we ensure that these data are available? Here we examine the effectiveness of four approaches to data archiving: no stated archiving policy, recommending (but not requiring) archiving, and two versions of mandating data deposition at acceptance. We control for differences between data types by trying to obtain data from papers that use a single, widespread population genetic analysis, STRUCTURE. At one extreme, we found that mandated data archiving policies that require the inclusion of a data availability statement in the manuscript improve the odds of finding the data online almost a thousand-fold compared to having no policy. However, archiving rates at journals with less stringent policies were only very slightly higher than those with no policy at all. At one extreme, we found that mandated data archiving policies that require the inclusion of a data availability statement in the manuscript improve the odds of finding the data online almost a thousand fold compared to having no policy. However, archiving rates at journals with less stringent policies were only very slightly higher than those with no policy at all. We also assessed the effectiveness of asking for data directly from authors and obtained over half of the requested datasets, albeit with about 8 days delay and some disagreement with authors. Given the long term benefits of data accessibility to the academic community, we believe that journal based mandatory data archiving policies and mandatory data availability statements should be more widely adopted.
Data collection 69
The study identified eligible STRUCTURE papers across journals with no, non-mandatory, or mandatory data-archiving policies, then checked 229 papers for publicly available genotype data. Mandatory JDAP journals were analyzed after policy implementation, with two also requiring data availability statements.
- Data collection: Papers using DNA sequence data were excluded because preparing raw sequence data for STRUCTURE re-analysis was considered very time consuming.The paper used Web of Science and journal-website searches to identify papers published in 2011 or 2012 that cited or used STRUCTURE.
- Journal policy categories: The sample covered four journals with no stated archiving policy, four with non-mandatory policies, and four with mandatory JDAP policies.Non-mandatory policies included recommendations or conditional sharing requirements, whereas JDAP required supporting data to be archived in an appropriate public archive.
- Journal policy categories: Two mandatory-policy journals, Molecular Ecology and Heredity, additionally required a data availability statement in each accepted manuscript.These statements described where publicly available data could be found, typically using database and accession numbers.
Statistical analysis 127
Statistical analyses found that archiving policies significantly improved online data availability overall, but the strongest effect came from mandatory archiving combined with a required data accessibility statement. Recommending archiving produced only a marginal, nonsignificant improvement over having no policy.
- Overall policy effect: Any archiving policy increased the odds of finding data online about 25 times compared with no policy.The association was significant (likelihood ratio test statistic = 4.27, p= 0.038; 95% confidence interval: 1.5 to 416.7).
- Policy-type comparisons: 3.6 times higher odds under a recommendation policy were not significant compared with no policy.The 95% confidence interval was 0.96 to 13.6, and the increase was only marginally more effective than having no policy.
- Policy-type comparisons: 17 times higher odds of online data availability were found for journals mandating archiving without requiring a data accessibility statement.This odds ratio was significantly greater than 1.
- Policy-type comparisons: 974 times higher odds of finding data online were associated with mandatory archiving plus a required data accessibility statement, compared with no policy.The 95% confidence interval was 97.9 to 9698.8, and this combination was more effective than every other policy type.
Requesting data directly from authors 170
Direct requests to authors yielded data for 59% of papers, but delivery averaged 7.7 days and sometimes required repeated follow-up. Some authors did not respond, data had been lost, or were upset by the requests.
- Results: All returned datasets arrived within 21 days, with BMC Evolutionary Biology deliveries taking eight to 14 days.For PLoS One, 10 datasets arrived within a week, 13 within eight to 14 days, and four within 15 to 21 days.
- Limitations: Requesting data by email upset some authors, particularly when reminded of journal archiving policies or contacted repeatedly.Online data were generally obtainable within minutes, unlike requested datasets.
- Results: 59% of requested datasets were returned, averaging 7.7 days to arrive.Data came from seven of 12 eligible BMC Evolutionary Biology papers and 27 datasets from 45 PLoS One papers.
Conclusions 209
Journal-based mandatory archiving policies, especially those requiring data accessibility statements, were highly effective at making research data available, whereas recommendation-only policies performed little better than no policy. Direct author requests can recover data but are delayed and vulnerable to disagreement and data loss, reinforcing the value of publicly supported archives.
- Policy effectiveness: Mandated archiving policies requiring a data accessibility statement were highly effective in ensuring research data availability.The conclusions identify the manuscript statement requirement as a key feature of effective journal-based archiving.
- Policy effectiveness: 10 of 89 datasets were available under recommendation-only policies, and no policy type in this group exceeded 23% online availability.Recommendation-only policies showed no significant advantage over having no policy at all.
- Policy effectiveness: 11% of datasets were online under PLoS One’s comprehensive policy versus 7% under BMC Evolutionary Biology’s simple request.The comprehensive policy was only marginally more effective than the simple request for freely available data.
- Policy effectiveness: 3.2 times more data were found online for journals with an Impact Factor of 5.0 than for journals with an Impact Factor of 2.2.Despite this association, the authors report that higher Impact Factor does not primarily explain the high archiving rate in mandatory-policy journals.
- Access and sustainability: Direct author requests can provide data but are delayed, may generate disagreement, and become less reliable as time since publication increases.Public databases require ongoing financial support, but the authors judge archiving costs highly cost effective because accessible data support reuse and checking.