Source-linked AI summary
Good Enough Practices in Scientific Computing
Greg Wilson, Jennifer Bryan, Karen Cranston, Justin Kitzes, Lex Nederbragt, Tracy K. Teal
TL;DR
Researchers often lack basic computing practices, leaving research workflows slow, difficult to reproduce, and difficult to assess for reliability. The paper synthesizes accessible practices for data, software, collaboration, organization, and tracking work, and concludes that they improve individual productivity and reproducibility. It also identifies practical boundaries where collaborators or reuse requirements make some recommendations difficult to apply.
Problem
Most researchers are not taught basic research-computing skills, contributing to slow, irreproducible work whose reliability they may not know.
Method
The paper synthesizes accessible “good enough” practices from published sources, author experience, and experiences from thousands of workshop participants.
Results
The practices are pragmatic and accessible to computing novices, apply to individuals and groups, and make researchers more productive while making computational work more reproducible.
Takeaways & Limitations
Researchers can adopt a minimum set of practices across data management, software, collaboration, project organization, tracking work, and manuscripts.
Takeaways & Limitations
Desktop-editor collaborators may require manuscript conversion, which can force manual merging because tools cannot always merge changes across formats automatically.
Abstract
from arXiv · showhide
We present a set of computing tools and techniques that every researcher can and should adopt. These recommendations synthesize inspiration from our own work, from the experiences of the thousands of people who have taken part in Software Carpentry and Data Carpentry workshops over the past six years, and from a variety of other guides. Unlike some other guides, our recommendations are aimed specifically at people who are new to research computing.
Author Summary
The paper identifies accessible computing practices that address inefficient, irreproducible, and poorly understood research workflows. It emphasizes incremental data management, tidy data, documented processing, and retained intermediate products.
- Researchers often lack basic research-computing training, which can make work slower, irreproducible, and difficult to assess for reliability.
- “Good enough” practices target researchers working alone or with small groups on projects lasting days to months.Practices are selected partly because large numbers of researchers continue using them after first trying them.
- Data Management: Data management should preserve raw data, improve readability, and retain intermediate products rather than overwriting or hiding them.Explicit intermediate files make pipelines easier to rerun, understand, share, describe, and modify.
- Data Management: Tidy data makes analysis easier by representing each variable in its own column and each observation in its own row.Wide tables can be gathered into measurement and time-point variables.
- Data Management: Every data-processing step should be scripted when feasible or documented precisely when manual actions are unavoidable.Documented steps support repeatable analyses, while scripts also make preparation easier to redo and reuse.
2 Software
The software recommendations adapt engineering practices for researchers, emphasizing modular, readable, reusable, testable, and shareable code. They focus on practices that help both current users and future reusers understand and run programs.
- Modular code built from short, single-purpose functions with clear inputs and outputs supports readability, reusability, and testability.
- Researchers should document programs with usage examples, decompose them into manageable functions, and avoid global variables.Functions are recommended to be no more than about 60 lines and take no more than half a dozen parameters.
- Functions, data structures, meaningful names, and explicit dependencies reduce duplication and make code easier to read and maintain.Dependencies can be recorded in a requirements.txt file or a Getting Started section.
- Researchers should test and share software as research products, including through reputable DOI-issuing repositories.Repositories such as Figshare and Zenodo can provide software DOIs.
- Programs should use program logic rather than commenting and uncommenting code to control behavior.A simple example or test dataset can check known outputs across machines and after changes.
3 Collaboration
The collaboration practices make projects easier for existing and new collaborators to understand, set up, and contribute to. They use documentation, shared task lists, explicit licensing, and citation information to support reuse.
- Projects should provide a README and CONTRIBUTING file describing their purpose, contacts, setup requirements, tests, and contribution guidance.
- A shared public to-do list should describe tasks clearly and can identify accessible starting points for newcomers.
- Explicit licenses clarify reuse rights for project software, data, and manuscripts.The paper recommends permissive licenses, including Creative Commons options for data and text and MIT, BSD, or Apache licenses for software.
- The paper cautions that “no commercial use” licenses may impede some forms of reuse.It recommends permissive software licenses because they are easier to integrate into other projects.
- A CITATION file should explain how to cite the project and locate citation information for artifacts with their own DOIs.
4 Project Organization
Project organization uses logical, consistent directories to help researchers and collaborators track, discover, and understand digital artifacts. Projects should be separated according to overlap in their data and code.
- Organizing project files in a logical, consistent directory structure helps people keep track of them.
- Each project should have its own directory named after the project, with boundaries chosen to support understanding.Researchers may define projects around manuscripts, shared themes, datasets, or algorithms.
- Work sharing no data or code is probably easiest to manage independently, while work sharing more than half is probably best managed together.Common code used across several projects should probably be placed in its own project.
2. Put text documents associated with the project in the doc
The project directory separates documentation, data, results, source code, and external executables so files remain understandable and reproducible. Clear names and an overview help people navigate the project and regenerate its outputs.
- Keep project documents and notebooks in the doc directory, with subdirectories for different file classes in large projects.
- Store raw data and metadata in data, generated intermediate and final outputs in results, and source code needed to regenerate results in src.
- Distinguish analytical code from controller scripts that combine functions, parameters, and inputs and execute the analysis from start to finish.
- Put external scripts and compiled programs in bin, but omit bin when the project has neither.
- Name files after their content or function rather than sequential numbers or manuscript locations, which may change as the project evolves.
- A README, CITATION, and LICENSE at the project root provide overview, attribution guidance, and licensing information.
5 Keeping Track of Changes
The paper recommends tracking project changes systematically through either manual versioning or version control. Version control reduces self-discipline demands, but its benefits depend on file type and size.
- Reference or retrieve specific project versions to support reproducibility before publication, during review responses, and when sharing supporting information.
- Version Control Systems: Version control records who changed what and when, synchronizes collaborators’ changes, and automates or enforces parts of the manual process.
- Researchers should choose either a systematic manual process or version control, depending on their readiness to adopt the tools.
- Manual Versioning: Manual versioning uses dated change logs and complete project copies, but collaborators must coordinate simultaneous edits and merge separate logs.
- What Not to Put Under Version Control: Version control works best for plain text; binary files lack precise diffs, and row or column reordering can appear as major changes in tabular data.
- What Not to Put Under Version Control: Small data and results may still be versioned for collaborator access and comparison across versions, while reproducible intermediates can be regenerated instead.
- What Not to Put Under Version Control: Large data or results files should not be included because current version control systems are not designed for megabyte- or gigabyte-sized files.
6 Manuscripts
The paper treats manuscript writing as a collaborative computing workflow rather than an email-based sequence of document merges. Groups should agree on a workflow and use tools that preserve a shared, reproducible master.
- Writing creates substantial project work, yet academic email workflows produce many files and tedious manual merging of coauthor feedback.
- The recommended alternatives aim to keep text accessible, prevent lost or overwritten work, combine contributions, avoid duplication, regenerate final forms, and share submissions.
- Groups should agree on a workflow and one feedback method before writing begins, whether email, an issue tracker, or a shared to-do list.
- Single Master Online: Online authoring tools keep everyone’s changes in one place and avoid manually merging them.
- Single Master Online: Converting online manuscripts to desktop formats can require manual merging because tools may not automatically reconcile changes across formats.
- A second approach treats papers like software, applying software-oriented practices to manuscript development.
1. Write the manuscript in a plain text format that permits version
The paper presents two manuscript workflows: plain-text files under version control for reproducibility, and online collaborative editing for accessible teamwork. The choice balances automation and reliability against collaborators’ familiarity and the overhead of specialized tools.
- Plain-text workflow: Plain-text manuscripts can be converted into formats such as PDF with scriptable tools like Pandoc.
- Plain-text workflow: Version control helps researchers find and merge concurrent changes while providing a platform for comments and review.
- Plain-text workflow: The version-control approach reuses data- and software-management skills and supports fully reproducible research.
- Plain-text workflow: The plain-text workflow requires contributors to understand markdown or LaTeX, make, BiBTeX, and Git/GitHub.
- Why Two Recommendations for Manuscripts?: Reviewers argued that this toolchain creates too high a collaboration barrier for colleagues outside the scientific computing community.
- Why Two Recommendations for Manuscripts?: Google Docs supports easy sharing, simultaneous editing, commenting, and replies to comments, whereas text-based systems can approximate these functions with more effort.
- Why Two Recommendations for Manuscripts?: Reproducible research and version control have fundamentally distinct goals, although version control can help achieve reproducibility.
- Why Two Recommendations for Manuscripts?: The paper therefore recommends online collaborative editing while discouraging desktop tools such as LibreOffice and Microsoft Word because they make collaboration harder.
7 What We Left Out
The paper omits practices whose benefits depend on project scale, prior expertise, or substantial investment. It favors simpler alternatives when they provide adequate functionality for newcomers and small research projects.
- Many advanced tools were left out because they build on core practices or require larger investments before paying off.
- Version control: Branching adds complexity for newcomers and can be adopted after mastering a basic edit-commit workflow.
- Build tools: Make can regenerate affected analyses, tables, plots, and papers, but newcomers can instead rerun everything with shell scripts.
- Testing: Unit tests prevent regression in large libraries and programs but are usually not compelling for solo exploratory work.
- Continuous integration: Continuous integration is invaluable in large projects with many contributors but is less suited to the paper’s intended audience.
- Performance: Profiling and performance tuning are worthwhile when program performance is a bottleneck, whereas most users spend more time getting programs correct.
- Coverage: Coverage tools become worthwhile as projects grow larger and are therefore not recommended here.
- The Semantic Web: Formal data ontologies are useful, but even simplified standards such as Dublin Core are rarely encountered in practice.
Conclusion
The paper offers pragmatic computing practices accessible to novices and applicable to individuals or groups. It argues that these practices improve productivity and reproducibility, while training support from institutions is needed for broader progress.
- The practices are based on the authors’ collective experience and experience from thousands of researchers reached through Carpentry organizations.
- The practices are pragmatic, accessible to computing novices, and applicable to both individuals and groups.
- The practices make researchers more productive by helping them get more done in less time and with less pain.
- The practices accelerate research by making computational work more reproducible.
- Universities and funding agencies need to support training in these tools to improve confidence in computational results and enable faster progress on research questions.
Box 1: Summary of Practices
The summary recommends lightweight practices for preserving, organizing, documenting, testing, sharing, and reproducing scientific computing work. It emphasizes explicit data and code structure, recoverable changes, and accessible research products.
- Data: Save raw data, create analysis-friendly data, and record every step used to process data.
- Data: Separate data behind tables and figures into reusable materials and submit data to a reputable DOI-issuing repository.
- Software: Search for maintained software libraries, test them before reliance, and make dependencies and requirements explicit.
- Software: Use meaningful names, avoid commenting and uncommenting code to control behavior, and provide simple examples or test data.
- Sharing: Submit code to a reputable DOI-issuing repository and make projects citable with explicit licensing.
- Collaboration: Create project overviews and shared public to-do lists to coordinate work.
- Project organization: Organize each project into named directories for documentation, data, results, source code, external programs, and clearly named files.
- Change tracking: Back up work promptly, keep and share changes frequently, mirror project folders, and use checklists, changelogs, copies, or version control.
Box 2: How Version Control Systems Work
Version control systems preserve project snapshots while recording who changed what and when. They also detect conflicting edits, support synchronization, and help recover or merge work safely.
- A version control system stores project-file snapshots in a repository and records each change’s author and timestamp.
- Users modify working copies, then save changes to create permanent records or share work with colleagues.
- When people edit files simultaneously, the system detects collisions and requires conflicts to be resolved before recording changes.
- Repositories can synchronize with one another, reducing dependence on any single repository as a point of failure.
- Version control automatically timestamps and logs saved changes, preserves enough information to recreate old versions, and supports conflict identification and merging.It records what actually changed rather than relying on manual backup names or changelogs.