Source-linked AI summary
VASP on a GPU: application to exact-exchange calculations of the stability of elemental boron
Maxwell Hutchinson, Michael Widom
TL;DR
Exact-exchange makes density-functional calculations much more expensive, motivating GPU acceleration for electronic-structure workloads. The paper ports hybrid functionals in VASP to GPUs, analyzes performance, and applies the implementation to competing boron structures, finding strong acceleration and lower energy for symmetry-broken partial-occupancy β while leaving α-versus-β stability unresolved.
Problem
Exact-exchange calculations are computationally expensive, while closely competing boron structures require higher-accuracy energies than standard DFT may provide.
Method
The paper implements exact-exchange hybrid functionals in VASP on GPUs and applies them to competing α- and β-rhombohedral boron structures.
Results
GPU execution outperforms one CPU core by about an order of magnitude, while symmetry-breaking partial occupancy substantially lowers the energy of β-boron relative to fully occupied β.
Takeaways & Limitations
GPU acceleration makes exact-exchange calculations on moderately complex structures tractable and supports studying symmetry-broken boron structures.
Abstract
from arXiv · showhide
General purpose graphical processing units (GPU's) offer high processing speeds for certain classes of highly parallelizable computations, such as matrix operations and Fourier transforms, that lie at the heart of first-principles electronic structure calculations. Inclusion of exact-exchange increases the cost of density functional theory by orders of magnitude, motivating the use of GPU's. Porting the widely used electronic density functional code VASP to run on a GPU results in a 5-20 fold performance boost of exact-exchange compared with a traditional CPU. We analyze performance bottlenecks and discuss classes of problems that will benefit from the GPU. As an illustration of the capabilities of this implementation, we calculate the lattice stability α- and β-rhombohedral boron structures utilizing exact-exchange. Our results confirm the energetic preference for symmetry-breaking partial occupation of the β-rhombohedral structure at low temperatures, but does not resolve the stability of α relative to β.
I. INTRODUCTION
Exact-exchange calculations provide more accurate treatment of electron exchange but are orders of magnitude more expensive, motivating GPU acceleration for VASP. The paper ports hybrid HF-DFT calculations to GPUs and reports substantial speedups.
- Exact exchange treats electron exchange interactions without the corresponding approximation, but increases runtime by orders of magnitude.
- GPU systems offer many low-cost, low-power cores suited to parallel electronic-structure operations, including VASP’s Fourier-transform bottleneck.
- The paper ports hybrid HF-DFT functionals including exact exchange to a GPU implementation of VASP.
- Exact-exchange calculations are motivated by the need to resolve closely competing boron structures whose DFT energy differences may be too small for reliable identification.
- GPU acceleration reaches speedups of up to 20× over CPU execution in computationally demanding cases.
II. ELEMENTAL BORON
Elemental boron combines practical properties with many competing structures, making its low-temperature ground state difficult to identify. The study targets exact-exchange energies of complex α- and β-rhombohedral structures using GPU acceleration.
- Boron is lightweight, strong, semiconducting, and structurally complex because its polymorphs arrange icosahedral clusters differently.
- Competing boron structures lie close enough in energy that DFT approximations could affect identification of the true low-temperature state.
- Figure 1 distinguishes α- and β-rhombohedral boron by showing fully occupied and partially occupied crystallographic sites with separate colors.
- The study uses exact-exchange calculations because boron’s complex structures may contain 107 or more atoms in low-symmetry cells.
A. Structural stability
The relative stability of α- and β-rhombohedral boron remains unresolved because their energy differences are small and depend on structural occupancy and functional choice. Partial occupancy lowers the β-derived structure’s energy, but calculations give conflicting comparisons with α.
- Boron has many polymorphs built from 12-atom icosahedral clusters, with several forms potentially stabilized by kinetics or impurities.
- The α structure has 12 atoms per primitive cell, whereas β has 105 and was initially associated with higher temperatures.
- DFT calculations found α lower in energy than ideal β, but the difference ranged from 48 meV/atom in LDA to 26 meV/atom in PW91.
- A partially occupied β′ structure with 107 atoms reached ΔEβ′α=-0.86 meV/atom using PW91 GGA, favoring β′ over α.
- The same PW91-optimized structure exceeded α by Δβ′α=15 meV/atom using LDA, underscoring the unresolved functional dependence.
B. Setup of DFT calculations
The calculations compare four α- and β-rhombohedral structures with plane-wave PAW/PBE settings and controlled k-point and FFT-grid parameters. Benchmarking uses meshes selected to balance energy resolution and computational efficiency.
- The study compares hR12, hR12x8, ideal hR105, and symmetry-broken aP107 structures.
- Run times scale linearly with independent k-points for conventional DFT but quadratically for HF-DFT.
- The calculations use PAW potentials, PBE exchange-correlation, and a 319 eV plane-wave cutoff.
- Typical runs use Monkhorst-Pack k-point meshes and report NKPT as independent k-points and NGX as the FFT-grid line dimension.
- A 3×3×3 mesh is adequate for meV/atom energy differences, while benchmarking focuses on computationally smaller 2×2×2 meshes.
III. PORTING OF VASP
VASP's exact-exchange calculations use numerical operations similar to conventional calculations, making performance improvements potentially transferable beyond this less commonly used functionality.
- III. PORTING OF VASP: VASP is a widely used Fortran code for DFT quantum chemistry calculations.Its codebase includes contributions dating back to the 1980s and code written in English, French, and German.
- III. PORTING OF VASP: Exact-exchange calculations in VASP rely on less commonly used functionality based on Hartree-Fock principles.The functionality nevertheless consists of numerical operations similar to those in conventional calculations.
- III. PORTING OF VASP: Performance enhancements obtained for exact-exchange may transfer to more conventional portions of VASP.
A. CPU Performance Analysis
The CPU analysis identifies exact-exchange routines as the dominant computational workload and shows that BLAS and FFT operations make VASP a strong GPU-acceleration candidate.
- A. CPU Performance Analysis: The test platform combines a quad-core Intel workstation with Tesla C2075 and C2050 GPUs, while Blacklight provides a larger Nehalem-based CPU system.The workstation includes 12GiB of DDR3 memory; Blacklight uses 256 blades with two eight-core Xeon processors per blade.
- A. CPU Performance Analysis: VASP achieves only 1.0-1.75 GFLOPs, or about 9-16% utilization, on the CPU test cases.The estimate compares measured VASP floating-point performance with a single-core LINPACK peak of 11.19 GFLOPs.
- A. CPU Performance Analysis: The exact-exchange workload is concentrated in FOCK_ACC and FOCK_FORCE, which account for over 98% of floating-point operations and over 97% of run-time.FOCK_ACC applies the Fock exchange operator, whereas FOCK_FORCE calculates Hellmann-Feynman forces on ions.
- A. CPU Performance Analysis: FOCK_ACC is called once per electronic minimization step, while FOCK_FORCE is called more frequently and becomes the dominant routine despite requiring approximately half as much effort per call.
- A. CPU Performance Analysis: Within FOCK routines, BLAS uses 20-30% of CPU run-time and FFTs use 35-50%, with the remainder divided between data manipulation and bookkeeping.This dependence on BLAS and FFT makes VASP a prime candidate for GPU acceleration.
B. Our Implementation
The GPU implementation is designed as a compartmental proof of concept that intercepts VASP execution while accelerating nested k-point and band-index calculations and projecting full-run costs.
- B. Our Implementation: The implementation preserves VASP's abstractions and interfaces while intercepting normal execution at five points.The first two intercepts create and destroy library contexts and global variables at the beginning and end of the main routine.
- B. Our Implementation: A simple per-system benchmark selects when FFTs should be sent to the GPU by comparing CPU and GPU FFT run-times.The optimal threshold on the tested system was 283, and this decision does not apply to FFTs in FOCK_FORCE.
- B. Our Implementation: The exact-exchange calculations use four nested loops over two k-point indices and two band indices.These loops compute several quantities for every pair of bands in the structure.
- B. Our Implementation: Full-run time is projected from electronic and ionic minimization counts, corresponding time increments, and an Other category for remaining code.The projection method underestimates cost by neglecting non-FOCK per-minimization operations, which are described as minimal.
C. Performance Results
GPU acceleration produces large speedups in the exact-exchange routines, with the strongest gains in FOCK_ACC and FOCK_FORCE, while truncated-run timings can differ from projected full-run timings.
- C. Performance Results: The GPU runs 7x-21x faster than the CPU in FOCK_ACC and 3x-15x faster in FOCK_FORCE.For large structures, the remaining non-FOCK code also achieves up to 1.5x speedup.
- C. Performance Results: GPU utilization is evaluated by assuming that CPU and GPU versions perform the same computational work.
- C. Performance Results: Differences between Table V and Table IV, especially for hR105, arise because truncated runs increase the relative impact of non-minimization overhead.That overhead is negligible in the full-run projections shown in Table IV.
IV. CONCLUSIONS
The GPU implementation makes exact-exchange calculations tractable for complex boron structures while confirming symmetry-breaking effects in β-boron. However, the relative stability of α and β remains unresolved, and broader applicability is projected rather than demonstrated.
- Structural conclusions: Exact-exchange calculations show that symmetry-breaking partial occupancy substantially reduces β-boron’s energy, but the optimized hR141 realization remains unstable relative to α.The authors therefore require structural re-optimization before determining α–β relative energies within exact exchange.
- Structural conclusions: The remaining variability among density functionals suggests that an even higher level of theory, such as quantum Monte Carlo, may be needed to resolve boron’s low-temperature stable structure.The reported functional sequence appears to converge toward values closer to LDA than GGA, but does not settle the structural question.
- Performance conclusions: The GPU implementation outperformed one CPU core by about an order of magnitude, making exact-exchange calculations on complex boron structures feasible.For the hR105 test case with two k-points, 64 CPU cores were needed to match the GPU time-to-solution, while blacklight’s best result was only 3% shorter than two GPUs.
- Broader implications: The implementation directly demonstrates exact-exchange feasibility for structures containing hundreds of ions, with multi-node GPU clusters projected for larger systems.The authors identify structural transitions in pnictide superconductors as another potential application.
- Broader implications: GPU acceleration may improve general DFT by enabling order-of-magnitude gains at low cost and potentially increasing treatable structure size by a factor of 2 or three.The authors also project an order-of-magnitude increase in achievable molecular-dynamics simulation times.