Source-linked AI summary
More Bang for Your Buck: Improved use of GPU Nodes for GROMACS 2018
Carsten Kutzner, Szilárd Páll, Martin Fechner, Ansgar Esztermann, Bert L. de Groot, Helmut Grubmüller
TL;DR
The paper asks how to produce as much molecular-dynamics trajectory as possible for a fixed budget, benchmarks compute nodes, and relates throughput to cost. It finds that consumer GPUs improve performance-to-price ratios, with the first consumer GPU increasing the ratio by a factor of 2–3, while noting benchmark limitations and legacy-GPU constraints.
Problem
The central question is how to produce as much molecular-dynamics trajectory as possible for a fixed budget.
Method
The study benchmarks GROMACS performance across diverse compute nodes and relates simulation throughput to node costs.
Results
Consumer GPUs improve the performance-to-price ratio by a factor of 2–3 when the first consumer GPU is added to a node.
Takeaways & Limitations
Cost-efficiency is maximized through hardware specialization, particularly by using consumer GPUs.
Takeaways & Limitations
Benchmark measurements share the last-level cache between two CPU co-located runs, so they are not equivalent to disabling unused CPU cores at constant frequency.
Abstract
from arXiv · showhide
We identify hardware that is optimal to produce molecular dynamics trajectories on Linux compute clusters with the GROMACS 2018 simulation package. Therefore, we benchmark the GROMACS performance on a diverse set of compute nodes and relate it to the costs of the nodes, which may include their lifetime costs for energy and cooling. In agreement with our earlier investigation using GROMACS 4.6 on hardware of 2014, the performance to price ratio of consumer GPU nodes is considerably higher than that of CPU nodes. However, with GROMACS 2018, the optimal CPU to GPU processing power balance has shifted even more towards the GPU. Hence, nodes optimized for GROMACS 2018 and later versions enable a significantly higher performance to price ratio than nodes optimized for older GROMACS versions. Moreover, the shift towards GPU processing allows to cheaply upgrade old nodes with recent GPUs, yielding essentially the same performance as comparable brand-new hardware.
1 Introduction
The paper asks how to maximize molecular-dynamics trajectory production under a fixed budget by specializing hardware for GROMACS. It updates earlier GROMACS 4.6 findings with current hardware and GROMACS 2018 developments, emphasizing GPU-oriented configurations and performance-to-price efficiency.
- Motivation: MD simulations can require substantial compute resources because biomolecular systems involve millions of time steps and repeated trajectories.Ensemble methods increasingly use many shorter trajectories, making aggregate trajectory production an important use case.
- Research question: Hardware selection is difficult because available nodes differ in CPU cores, sockets, GPUs, memory, and interconnects.The paper frames these choices around maximizing computation performance relative to node price.
- Approach: The study specializes cluster design for MD and measures throughput as total trajectory length produced over a node’s lifetime for a fixed budget.GROMACS is chosen because it is fast, widely used, and freely available.
- Evaluation criteria: The study evaluates performance-to-price ratio while also considering energy consumption and rack-space requirements.Rack-space constraints are incorporated through hardware preselection and a minimum average density of one GPU per rack unit for server nodes.
- Scope: The evaluation prioritizes efficient aggregate throughput from concurrent simulations rather than minimizing the time to generate one individual trajectory.This scope reflects the trade-off between simulation rate and parallel efficiency in strong scaling.
- GROMACS 2018 developments: GROMACS 2018 shifts the optimal CPU/GPU balance toward GPUs by offloading PME and other work, enabling stronger GPUs and higher P/P ratios.The dual pair list algorithm also reduces short-range interaction calculations and CPU pair-search frequency.
- Node optimization: For whole-node efficiency, a consumer GPU with lower GPU-only P/P can be preferable when it delivers higher performance and fits node-level constraints.Among the cited examples, the GTX 1080 leads Pascal GPU P/P and the RTX 2070 leads Turing GPU P/P.
2 Methods
The study benchmarks GROMACS across hardware and software configurations using standardized environments, workload protocols, and performance-to-price evaluation criteria.
- Benchmark design: The main study reused two benchmark input files from the 2014 investigation to facilitate comparisons between new and old hardware.
- Software environment: GROMACS 2018 used AVX2 SIMD instructions, while builds and CUDA combinations varied between the main study and evaluation benchmarks.
- Software environment: All hardware was tested in a common software environment, with specified operating-system exceptions for selected nodes and evaluation benchmarks.
- Calibration: Older GCC/CUDA measurements were corrected by multiplying performance by 1.025 after identical-hardware MEM tests found differences of about 1.5%–4%.
- Performance measurement: The benchmark protocol ran N simulations on nodes with N GPUs, each using 1/N of available CPU resources, and reported aggregate performance.
- Performance measurement: Benchmark measurements excluded initial time steps to avoid non-optimal load-balancing effects and repeated GROMACS evaluations three times.
3 Results
GROMACS 2018 improves GPU-oriented simulation performance and shifts balanced configurations toward fewer or slower CPU cores. Consumer GPU nodes provide strong performance-to-price advantages, while upgrading existing nodes can be highly effective.
- GROMACS performance developments: 35–84% additional PME-offload improvements were observed with recent GPUs, whereas offloading to the Tesla K80 caused a slowdown on legacy hardware.
- GROMACS performance developments: 4–6 slower server cores were usually sufficient with GROMACS 2018 to exceed 80% of peak simulation performance.
- CPU-GPU balance: 10–15 core-GHz generally sufficed to approach peak performance with mid- to high-end GPUs in the tested biomolecular workloads.
- Hardware ranking: Consumer GPU nodes had P/P ratios about 3–6 times higher than CPU-only counterparts.
- Hardware ranking: The best current P/P ratio came from an E5-2630v4 or Ryzen 1950X with two or more RTX 2080 GPUs, starting at 3,000 € net.
- Upgrading existing nodes: Replacing old GPUs with recent models could yield a factor-of-3.5 MEM performance gain, while four 1080Ti GPUs performed similarly with new or older CPUs.
- Cost and energy efficiency: With GROMACS 2018 and current hardware, trajectory costs on the best GPU nodes were about 0.3 times those of CPU-only counterparts.
4 Conclusions for GROMACS 2018
Consumer GPU nodes deliver substantially more trajectory output per invested Euro than CPU-only or professional-GPU nodes, while GROMACS 2018 shifts the optimal CPU/GPU balance further toward GPUs. PME offload also makes upgrading existing nodes with modern GPUs a cost-effective way to restore that balance.
- Consumer GPU nodes provide significantly higher trajectory output per invested Euro than nodes without GPUs or with professional GPUs.
- Today, raw node prices yield three to six times more trajectory on consumer GPU nodes, compared with two to three times in 2014 using GROMACS 4.6.
- Including energy and cooling costs increases the consumer-GPU advantage from a factor of two to about three.
- PME offloading moves the optimal hardware balance even more toward consumer GPUs.
- Optimal CPU/GPU combinations require about four to eight CPU cores per 1080Ti or 2080; 10–15 core-GHz are sufficient and 15–20 core-GHz are future-proof for similar systems.
- Replacing old GPUs with state-of-the-art models can restore the optimal GROMACS 2018 CPU/GPU balance at the comparatively small cost of GPUs alone.
5 Outlook
The authors expect continued hardware evolution to favor configurations with stronger GPUs and few CPU cores, and report that the findings transfer well to GROMACS 2019. The 2019 release adds OpenCL PME offload and CUDA bonded-interaction offload, with the latter benefiting mainly CPU-limited or bonded-work-heavy simulations.
- Future hardware configurations with only a few CPU cores paired with fast GPUs may achieve still higher performance-to-price ratios.
- Readers with uncovered hardware configurations are encouraged to run the CC-licensed benchmark inputs so updated tables can include additional data.
- The presented results transfer very well to the GROMACS 2019 code.
- GROMACS 2019 introduces PME offload support using OpenCL, particularly useful on AMD GPUs.
- Radeon Vega GPUs compare favorably to competing GPUs, especially in performance-to-price ratio against Tesla GPUs.
- The 2019 release can offload most bonded interactions with CUDA, but the performance benefit is expected mainly when CPU resources are low or bonded work is substantial.