Source-linked AI summary
Best Practices for Scientific Computing
Greg Wilson, D. A. Aruliah, C. Titus Brown, Neil P. Chue Hong, Matt Davis, Richard T. Guy, Steven H. D. Haddock, Katy Huff, Ian M. Mitchell, Mark Plumbley, Ben Waugh, Ethan P. White, Paul Wilson
TL;DR
Scientists often lack training in efficient scientific software development despite spending increasing time building and using software. This paper presents research- and experience-based best practices, concluding that they improve programming productivity, code reliability, and confidence in computational results.
Problem
Scientific programmers face repetitive work and evolving requirements, while lacking broadly taught practices for producing readable, reliable, and maintainable software.
Method
The paper outlines 25 research- and experience-based recommendations covering human-readable code, automation, incremental development, defensive programming, documentation, and optimization.
Results
The practices improve scientific programming productivity and resulting code reliability, while supporting reproducible computational research.
Takeaways & Limitations
Adopting these tools and approaches can increase confidence in computational results and requires training and institutional investment.
Abstract
from arXiv · showhide
Scientists spend an increasing amount of time building and using software. However, most scientists are never taught how to do this efficiently. As a result, many are unaware of tools and practices that would allow them to write more reliable and maintainable code with less effort. We describe a set of best practices for scientific software development that have solid foundations in research and experience, and that improve scientists' productivity and the reliability of their software.
1 Write programs for people, not computers.
Scientific software should be written for human readers as well as correct execution, because readability helps programmers verify intent and work productively. The section recommends limiting memory demands, using meaningful names, and maintaining consistent formatting and style.
- Readable code helps programmers understand whether software does what it is intended to do and supports productive development.Human working memory is limited and pattern matching abilities must be considered when writing programs.
- Programs should require readers to remember only a handful of facts or chunks at once.Grouping rectangle coordinates into two points makes a function easier to read and remember than taking four separate coordinates.
- Names should be consistent, distinctive, and meaningful rather than vague or confusingly similar.Names such as a, foo, results, and results2 can cause confusion.
- Consistent code style and formatting reduce reading time and mistakes.Inconsistent indentation, capitalization, or naming conventions makes code more difficult to read.
2 Let the computer do the work.
Scientific work contains repetitive computational tasks that scientists should automate through scripts, command histories, and workflow build tools to save time and reduce errors. Reproducibility also requires automatically recording machine-readable data provenance, including identifiers, versions, parameters, and software information needed to recreate outputs.
- 2 Let the computer do the work.: Repetitive manual computation wastes time and can cause errors when researchers lose focus.Examples include processing many data files identically and regenerating figures after adding new data.
- 2 Let the computer do the work.: Scientists should use scripts and command histories to repeat tasks, saving time and reducing errors.Command-line history tools display and reexecute recent commands with minor edits to filenames or parameters.
- 2 Let the computer do the work.: Build tools should automate workflows by encoding dependencies so one command regenerates every outdated intermediate file and final result.This avoids errors and inefficiencies caused by repeating commands manually.
- 2 Let the computer do the work.: Reproducible outputs require automatically recording machine-readable provenance containing identifiers, versions, generation parameters, and software names.These records capture what is needed to re-create outputs and can be read by other programs.
3 Make incremental changes.
Scientific programmers should make small, feedback-driven changes because requirements often evolve as results emerge. Version control supports collaboration, conflict resolution, history, provenance, and reproducibility by preserving manually created project materials.
- Evolving requirements: Because requirements often emerge from results, scientists should avoid design approaches that depend on specifying them fully in advance.Scientific requirements are rarely frozen, and scientists may not know what a program should do next until its current version produces results.
- Small steps and feedback: Programmers are most productive when they work in roughly hour-long steps with frequent feedback and course correction, grouped into iterations lasting roughly one week.This approach accommodates cognitive constraints and the reality that requirements change during development.
- Version control: Version control replaces fragile file-sharing practices by storing project snapshots that programmers can modify, commit, and share.It addresses challenges in tracking changes, reverting mistakes, and collaborating on code and data.
- Version control: Version control highlights simultaneous edits, requires conflict resolution, preserves complete file histories and metadata, and supports provenance for code and data.Stored history enables arbitrary versions to be retrieved and compared, while metadata records changes and authorship.
- Reproducibility: Reproducibility is maximized by version-controlling manually created materials, including programs, field observations, and paper sources, while regenerating automated outputs when needed.Binary files may instead be archived, with metadata describing their contents kept in version control.
4 Don’t repeat yourself (or others).
The DRY Principle improves maintainability and reliability by keeping data and code from being repeated. It calls for authoritative data representations, modularized code, and reuse of established software rather than rewriting solutions.
- Repeated data or code is harder to maintain because changes must be made in multiple locations, increasing errors and inconsistencies.The DRY Principle applies to both data and code.
- Each piece of data should have a single authoritative representation, including physical constants, raw data files, and geographic locations identified for lookup.This ensures the entire program uses the same values and canonical records.
- At small scales, modularizing code instead of copying and pasting reduces error rates and keeps fixes effective everywhere.Modularization also supports a single accurate mental model of the program’s functionality.
- Modularized code can be more easily repurposed for other projects.
- At larger scales, programmers should reuse established libraries or packages instead of rewriting solutions to well-established problems.Tens of millions of lines of high-quality open-source software are freely available, with at least as much available commercially.
5 Plan for mistakes.
Because mistakes are inevitable and maintaining code validity over time is challenging, the paper recommends layered defenses rather than any single practice. Defensive programming, automated testing, and debugging tools help detect, document, and investigate failures.
- Mistakes are inevitable, so several complementary practices should be used together to verify and maintain code validity over time.No single practice has been shown to catch or prevent all mistakes.
- Defensive programming uses assertions to verify valid inputs, consistent outputs, and other conditions during execution.Assertions can halt the program immediately when something goes wrong and serve as executable documentation of intended behavior.
- Automated tests cover individual units, integrated components, and regressions, and computers can rerun them whenever code changes.Off-the-shelf unit-testing libraries help initialize inputs, execute tests, and report results uniformly.
- Scientific tests compare outputs with simplified cases, experimental data, trusted earlier programs, or test cases derived from discovered bugs.Tests assess whether code matches expectations based on the researcher’s understanding of the problem.
6 Optimize software only after it works correctly.
Make software work correctly before optimizing it: determine whether speedup is worthwhile, then use profiling to locate actual bottlenecks. Scientists are generally most productive in the highest-level language possible, shifting to lower-level languages only when performance gains are needed.
- 6.1: Correctness comes first; optimize only when performance improvement is worthwhile and use a profiler to identify bottlenecks.The complexity of modern computers and software makes performance bottlenecks difficult to predict, even for experts.
- 6.2: Scientists are most productive in the highest-level language possible because programmers write roughly the same number of lines per unit time across languages.Lower-level languages require more lines of code for the same task.
- 6.2: Shift to low-level languages such as C and Fortran only after confirming that their performance boost is needed.This approach allows more code to be written and tested in the same amount of time.
- 6.2: Rapidly prototype in a high-level language before implementing in a necessary low-level language to evaluate designs early.This can help even when the need for a low-level language is known before coding begins.
8 Collaborate.
Collaboration practices improve scientific software by reducing bugs, improving readability, spreading knowledge, and coordinating work across changing or growing teams. The section recommends pre-merge code reviews, selective pair programming, and issue tracking to make collaboration effective.
- Code review: Code reviews reduce bugs, improve readability, spread knowledge, and promote good practices within a team.Research identifies code reviews as the most cost-effective way of finding bugs in code.
- Code review: Projects should require pre-merge code reviews because reviews not required for repository access will soon stop being performed.Reviews may occur before or after commits, but the recommendation is to make them a condition of merging code.
- Pair programming: Teams should use pair programming when bringing someone new up to speed or tackling particularly challenging work.The driver writes code while the navigator provides real-time feedback and tracks larger design and consistency issues; some programmers find the practice intrusive.
- Issue tracking: Issue tracking tools help growing teams avoid duplicated work and dropped tasks by maintaining lists of tasks and bugs.They also make it easier to transfer tasks between people; repository hosting services such as GitHub often include them.
9 Conclusion
The recommended practices improve scientific programming productivity and code reliability, supporting faster results, greater confidence, and reproducible computation. Their adoption should proceed incrementally, supported by training and institutional investment.
- 9 Conclusion: The practices improve scientific programming productivity and resulting-code reliability for individual and group work, increasing the speed of results and confidence in them.The paper presents them as prerequisites for reproducible computation.
- 9 Conclusion: 25 recommendations provide a beginning rather than an endpoint, with links to more advanced practices available through Software Carpentry.The recommendations are intended for individuals and groups that have incorporated them into their work.
- 9 Conclusion: The implementation time cost is almost immediately offset by programmers’ productivity gains, while mutually reinforcing practices reduce the total effort required.Despite these benefits, the recommendations may initially seem intimidating to implement.
- 9 Conclusion: Online tutorials, workshops, and classes can teach implementation, and this training has proven effective at driving adoption in scientific settings.Software Carpentry is cited as one example of a group organizing such training.
- 9 Conclusion: Scientists, universities, and funding agencies should adopt and support tools and approaches that improve software quality, production efficiency, confidence in computational results, and scientific progress.The paper frames this investment as necessary for computing to achieve the rigor expected elsewhere in science.