Source-linked AI summary
Quantum ESPRESSO toward the exascale
Paolo Giannozzi, Oscar Baseggio, Pietro Bonfà, Davide Brunato, Roberto Car, Ivan Carnimeo, Carlo Cavazzoni, Stefano de Gironcoli, Pietro Delugas, Fabrizio Ferrari Ruffino, Andrea Ferretti, Nicola Marzari, Iurii Timrov, Andrea Urru, Stefano Baroni
TL;DR
Quantum ESPRESSO must adapt to heterogeneous accelerator-based architectures as energy constraints hinder progress toward exascale computing. The paper reviews a strategy combining sustainable refactoring, performance-portable libraries, and GPU ports, with reported speedups of 2× to 3× across varied platforms and benchmarks. It also identifies memory and computation bottlenecks that constrain larger-scale use.
Problem
Energy constraints and increasingly heterogeneous architectures create a major porting and maintenance challenge for complex Quantum ESPRESSO community codes.
Method
The developers combine GPU-specific ports with layered refactoring and low-level libraries that encapsulate architecture-specific kernels and operations.
Results
2× to 3× speedup was consistently achieved for a CUDA Fortran strategy across varied platforms and benchmarks.
Takeaways & Limitations
Performance portability is pursued by moving optimized and specialized computations into shared libraries that can benefit multiple Quantum ESPRESSO codes.
Takeaways & Limitations
Larger-scale calculations still require work to identify and remove memory and computation bottlenecks, including limits from GPU-card memory.
Abstract
from arXiv · showhide
Quantum ESPRESSO is an open-source distribution of computer codes for quantum-mechanical materials modeling, based on density-functional theory, pseudopotentials, and plane waves, and renowned for its performance on a wide range of hardware architectures, from laptops to massively parallel computers, as well as for the breadth of its applications. In this paper we present a motivation and brief review of the ongoing effort to port Quantum ESPRESSO onto heterogeneous architectures based on hardware accelerators, which will overcome the energy constraints that are currently hindering the way towards exascale computing.
I. INTRODUCTION
The paper reviews Quantum ESPRESSO’s ongoing adaptation to heterogeneous architectures, motivated by exascale computing opportunities and unprecedented software challenges. It covers performance portability, sustainable restructuring, GPU capabilities, and benchmark-based bottleneck analysis.
- Motivation: The manuscript focuses on porting Quantum ESPRESSO to heterogeneous architectures that combine multiple cores with hardware accelerators.The goal is to address high performance alongside acceptable energy consumption.
- Challenges and opportunities: Heterogeneous architectures offer a path toward exascale computing but require deeper software changes than earlier hardware transitions.The paper describes both unprecedented opportunities and challenges.
- Performance portability: Performance portability seeks comparable performance across architectures while minimizing hardware-specific code maintenance.The effort begins with low-level libraries but also requires higher-level restructuring for long-term maintainability.
- Sustainable development: Sustainable development restructures codes to make them easier to maintain, extend, and port to future architectures.This complements low-level library work within the broader modernization effort.
- GPU evaluation: NVIDIA GPU benchmarks use relatively small machines to identify bottlenecks, inefficiencies, and calculation sizes that saturate GPU computational power.They are not intended to demonstrate performance on large systems.
- Conclusions: The conclusion analyzes achieved results and outlines forthcoming actions.
II. QUANTUM ESPRESSO AT THE TURN OF THE TWENTIES
Quantum ESPRESSO is an open-source, modular distribution for first-principles materials simulations that has expanded its methods, property calculations, interoperability, and HPC support. Its scope includes ground-state, molecular-dynamics, response, many-body, and diverse materials-property applications.
- Core capabilities: The distribution combines PWscf, CP, and FPMD capabilities for self-consistent calculations, structural optimization, lattice dynamics, and first-principles molecular dynamics.The package lineage includes density-functional theory, plane waves, and pseudopotentials.
- Methods: Quantum ESPRESSO has implemented advanced methods including density-functional perturbation theory, projector-augmented waves, spin-orbit coupling, and several functional corrections.The text highlights linear-response innovation and expanding theoretical scope.
- Project structure: Quantum ESPRESSO is an open-source distribution conceived as an integrated suite of loosely coupled packages rather than a monolithic application.This structure supports collaboration, code sharing, and innovation in methods and algorithms.
- Applications: Its property-calculation portfolio covers reaction barriers, superconductivity, electron-phonon effects, NMR, EPR, transport, spectroscopy, solvation, and electrochemical models.
- HPC support: Computational development has pursued MPI and OpenMP parallelization for modern and massively parallel HPC architectures.
- Interoperability: The project supports external workflows as a quantum engine and produces DFT data for quantum Monte Carlo, many-body, Wannier, transport, and data-analysis codes.
III. CHALLENGES AND OPPORTUNITIES OF NEW HETEROGENEOUS ARCHITECTURES
Heterogeneous exascale systems shift computational power toward accelerators while introducing difficult requirements for data movement, memory, fault tolerance, and programming models. Quantum ESPRESSO responds through layered refactoring, separation of concerns, and parallel GPU porting tracks.
- Architecture shift: Most computational power in future exascale machines is expected to come from accelerators.This follows a growing ratio of accelerator power to traditional CPU power at the single-node scale.
- Architecture challenges: Heterogeneous systems demand major redesigns for data movement, heterogeneous memory management, fault tolerance, and programming paradigms.Extremely parallel applications must handle data movement between disjoint memory spaces.
- Layered refactoring: Quantum ESPRESSO is being refactored into weakly coupled modules and libraries because extensive recoding for each architecture is no longer sustainable.The distribution’s complexity and rapidly changing hardware motivate independent maintenance of components.
- Software layers: The proposed structure separates property calculators, quantum-engine modules, mathematical libraries, and low-level system libraries.Property calculators target materials properties, while quantum-engine modules solve the one-particle Schrödinger equation and related tasks.
- Separation of concerns: Separation of concerns assigns architectural details to low-level and HPC developers while method developers focus on property calculators.The MAX Centre of Excellence uses this principle to support porting quantum-materials codes.
- Implementation strategy: The developers pursue converging tracks: porting pw.x and other components to NVIDIA GPUs while refactoring the distribution into stand-alone modules and libraries.The intended endpoint is gradual encapsulation of architecture-specific kernels in low-level libraries, using OpenMP-5 features.
A. Performance portability
Quantum ESPRESSO’s performance-portability effort abstracts accelerator-specific operations through shared libraries and APIs. DevXlib and UtilXlib expose memory, linear-algebra, synchronization, and parallelization facilities while limiting disruption to scientific code.
- Motivation: Performance portability remains difficult because general-purpose frameworks lack maturity and widespread adoption, while Fortran support is limited or absent.The MAX Centre of Excellence coordinates a collective effort across materials-modeling applications.
- DevXlib: DevXlib is progressively entering the GPU port to hide CUDA Fortran extensions and provide a more portable abstraction.It originated as a shared initiative among MAX Centre codes, including Quantum ESPRESSO and Yambo.
- Design goals: The portability effort targets Fortran support, multiple hardware and software stacks, device-code encapsulation, and limited source disruption.
- Abstract interfaces: Device-agnostic APIs expose memory copying and synchronization so scientific developers can control required operations without depending on a specific backend.
- Library capabilities: DevXlib provides interfaces for memory buffers, linear algebra, domain-specific operations, and Fortran-compatible device data structures.Its operations include memory handling, BLAS/LAPACK-like routines, and specialized plane-wave kernels.
- Supporting infrastructure: UtilXlib supplies common MPI, OpenMP, timing, error, and memory utilities, extended with GPU memory-space execution and synchronization points.
B. Towards a sustainable development, maintenance, and porting model
QUANTUM ESPRESSO’s growth has made maintenance and extension increasingly challenging, motivating modern software-engineering practices. The development model combines testing, interoperability, modularization, and API-based code layers to improve sustainability and portability.
- At version 6.5, QUANTUM ESPRESSO comprised almost 600,000 lines of Fortran and 60,000 lines of C, Python, and shell code, plus tests, examples, and documentation.
- The project introduced automated tests, structured XML and HDF5 I/O, and code organization into modules and libraries.
- Libraries encapsulate internal data structures and exchange data through predefined APIs, allowing independent development, distribution, compilation, and linking.
1. Domain-specific mathematical libraries
QUANTUM ESPRESSO separates major computational functions into domain-specific libraries to support performance portability and reuse. These libraries expose architecture-transparent interfaces while accommodating optimized implementations for heterogeneous systems.
- Three major stand-alone libraries were extracted: LAXlib for dense-matrix operations, FFTXlib for distributed three-dimensional fast Fourier transforms, and KS_Solvers for iterative diagonalization.
- Because these libraries perform much of a typical electronic-structure calculation, optimizations made within them benefit multiple codes and computations.
- Separating concerns at the library level supports performance portability across heterogeneous architectures, extending the original MPI and OpenMP CPU approach.
- LAXlib and FFTXlib encapsulate their data structures and provide interfaces transparent to the underlying architecture, enabling third-party use.
- The Electronic Structure Library initiative provided related goals and demonstrated decoupling of KS_Solvers, LAXlib, and FFTXlib from their native codes.
2. Quantum-engine modules
Quantum-engine modules reorganize reusable functionality that is difficult to encapsulate as independent libraries. Modules supports core Kohn–Sham operations, while LR_Modules unifies common linear-response and many-body perturbation-theory functionality for reuse across packages and codes.
- General modules retain the suite’s global data structure where library-style encapsulation would be difficult, inefficient, or unfeasible.
- Modules: Modules implements operations for self-consistent iterative Kohn–Sham calculations, including lattices, symmetries, exchange-correlation potentials, plane waves, k points, and pseudopotentials.
- LR_Modules: LR_Modules was created to unify, harmonize, generalize, and refactor functionality shared by QUANTUM ESPRESSO’s linear-response and MBPT codes.
- LR_Modules: LR_Modules supplies response quantities, iterative solvers, exchange-correlation kernels, symmetrization routines, and empty-state projectors.
- Applications: The modules support applications including phonons, optical absorption and collective excitations, electron-phonon coupling, transport, and superconducting properties.
- Reuse: Refactored LR_Modules routines can be used in future QUANTUM ESPRESSO linear-response or MBPT codes and in third-party codes.
3. Interoperability
QUANTUM ESPRESSO uses standard XML and HDF5 formats to make output structured, machine-processable, and accessible to external tools. Separate packages provide schema-aligned bindings, file APIs, converters, and Python access to postprocessing utilities.
- XML and HDF5 encode data hierarchy and types, supporting automatic processing by external applications and postprocessing with Python.
- Python tools generate Fortran XML read/write bindings from schemas and keep the generated routines aligned with the format specifications.
- The qeschema package provides APIs for XML, HDF5, and UPF files and converts selected structures and charge densities to formats used by external atomistic tools.
- The postQE package exposes PP postprocessing utilities to Python applications through extensions or output-file extraction tools for computationally demanding cases.
C. Evolution of the GPU-enabled version
QE-GPU evolved from an effective but maintenance-heavy CUDA C plugin approach into a broader GPU port combining CUDA Fortran, directives, and accelerated libraries. Benchmarks show numerical equivalence and speedups that depend strongly on workload size and data parallelism.
- The original CUDA C plugins improved performance but were difficult to maintain because of extensive replicated Fortran–CUDA interface code.
- QE-GPU was rewritten from scratch using CUDA Fortran, with the prior strategy demonstrating consistent 2× to 3× speedups across platforms and benchmarks.
- The directive-based approach can validate GPU code on CPUs and retain a single source, while explicit memory management exposes intra-node transfers and synchronization.
- Accelerated LAXlib and FFTXlib use architecture-specific APIs and libraries rather than directive-based acceleration.
- GPU acceleration covers self-consistency, Kohn–Sham Hamiltonian solution, atomic forces, exact exchange, and numerous Hamiltonian, preconditioning, and density-generation kernels.
- The GPU-enabled pw.x preserves CPU-compatible features and equivalent results, with maximum energy and force discrepancies of 2·10−8 Ry and 3·10−5 Ry/Bohr.
- The smallest benchmark, a 98-atom, 246-electron compound, achieved only 1.4x speedup, whereas larger problems reached speedups exceeding 3x.
IV. OUTLOOK AND CONCLUSIONS
Quantum ESPRESSO’s outlook centers on maintainable, performance-portable support for heterogeneous architectures. Ongoing work targets GPU integration, OpenMP-5 and ARM support, while recognizing memory and scaling constraints in large calculations.
- The project prioritizes modularity, portability, and performance because heterogeneous architectures make future algorithms and property calculations harder to implement and maintain.
- Merging the main distribution with the NVIDIA CUDA Fortran version will remove the need to keep separate versions aligned as the main code evolves.
- OpenMP-5 support is being introduced, Intel collaboration targets Ponte Vecchio compatibility, and ARM vectorized architectures are under development.
- Current benchmarks are realistic but relatively small, and larger calculations require further work on memory and computation bottlenecks.
- The data supporting the findings are openly available in the Materials Cloud Archive.