Source-linked AI summary
Beyond DFT electronic band structures via dual-space orbital learning
Minhyuk Kang, Beom Seok Kang, Nikolai Argatoff, Miguel Nouman, Danish Khan, William A. Goddard III, Anima Anandkumar
TL;DR
High-fidelity band structures are computationally demanding, and broad, state-resolved prediction across diverse materials remains an open gap. O-kRystal learns higher-level corrections from lower-cost PBE calculations and orbital information, achieving low HSE06 band-energy error across nearly 6,000 three-dimensional materials and extending to G0W0 quasiparticle energies.
Problem
Higher-fidelity electronic band calculations are substantially more expensive than semilocal DFT, while prior learning approaches had not jointly demonstrated broad material coverage, spin resolution, and variable band counts.
Source: §1 · pp. 2–3Method
O-kRystal uses lower-cost electronic structure as both a baseline and orbital representation, learning state-specific corrections from complementary orbital information.
Source: §1 · p. 3, §3 · p. 13Results
0.171 ± 0.009 eV band-energy MAE for PBE-to-HSE06 prediction within ±5 eV of the band-edge midpoint across nearly 6,000 three-dimensional materials, with a 68.7% MAE reduction over PBE on six strongly correlated materials.
Source: §1 · p. 3Takeaways & Limitations
The framework can recover material-specific dispersion changes without target-theory calculations at each queried k-point and may help prioritize candidates for more demanding calculations.
Source: §3 · p. 14O-kRystal still requires a converged baseline electronic-structure calculation and predicts target-theory state energies rather than wavefunctions.
Source: §3 · p. 14
Abstract
from arXiv · showhide
Accurate electronic band structures are essential for predicting material properties, but high-fidelity calculations are computationally demanding. We introduce O-kRystal, a dual-space orbital-learning framework that uses a lower-cost Perdew-Burke-Ernzerhof (PBE) calculation as both an energy baseline and a source of structured orbital information. Combining PBE band energies with real- and reciprocal-space orbital matrices, O-kRystal predicts higher-level corrections for individual electronic states as a function of crystal momentum and spin. For states within 5 eV of the band-edge midpoint, it predicts Heyd-Scuseria-Ernzerhof (HSE06) band energies with a mean absolute error of $0.171 \pm 0.009$ eV across nearly 6,000 three-dimensional materials spanning insulating, metallic, magnetic, and nonmagnetic systems. On a separate benchmark of six strongly correlated materials, it reduces the corresponding PBE band-energy MAE by 68.7% and resolves orbital-dependent changes in the relative placement of transition-metal d and ligand p states. Despite training only at high-symmetry points, it maintains comparable accuracy at unseen points along band paths and reconstructs HSE06 densities of states from dense Brillouin-zone predictions. Finally, training O-kRystal on $G_0W_0$ quasiparticle energies for two-dimensional materials demonstrates its applicability across levels of theory and the potential of orbital learning for higher-fidelity, momentum-resolved electronic-structure prediction beyond density functional theory.
1 Introduction
O-kRystal uses lower-cost PBE calculations and orbital information to predict higher-level, momentum-resolved band structures. It targets a gap in scalable prediction across diverse materials and reports accuracy on HSE06 and G0W0 benchmarks.
1 Introduction
- HSE06 and GW can improve electronic-state descriptions over semilocal DFT but are substantially more expensive, motivating correction models that reuse low-cost electronic information.The passage reports approximate O(N^4) scaling for conventional hybrid-functional and G0W0 implementations versus O(N^3) for Kohn–Sham DFT.Source: §1 · p. 2
- Prior approaches had not demonstrated higher-fidelity band prediction at scale while jointly covering diverse chemistry, metals and insulators, explicit spin, and variable band counts.Existing approaches were limited by settings such as non-magnetic materials, fixed band representations, or restricted material sets.Source: §1 · pp. 2–3
- O-kRystal learns state-specific corrections to PBE energies using complementary real- and reciprocal-space orbital representations, accommodating different valence and conduction band counts across materials.Real space represents localized orbital interactions, while reciprocal space exposes momentum dependence and electronic dispersion.Source: §1 · p. 3
- 0.171 ± 0.009 eV band-energy MAE for states within ±5 eV of the band-edge midpoint, across nearly 6,000 diverse three-dimensional materials, for PBE-to-HSE06 prediction.The evaluated materials include insulating, metallic, and spin-polarized electronic structures.Source: §1 · p. 3
- 68.7% reduction in band-energy MAE over PBE on six held-out strongly correlated materials, with orbital-projected spectra resolving distinct Mn 3d and O 2p energy corrections.For dense Brillouin-zone states within the 5 eV window, MAE falls from 1.532 eV with PBE to 0.479 eV.Source: §1 · p. 3
- High-symmetry-only training generalizes to unseen k-points, reconstructs HSE06 densities of states, and achieves 0.126 ± 0.017 eV quasiparticle-energy MAE on 286 two-dimensional semiconductors using G0W0 targets.The predicted band structures also improve experimentally validated fundamental-gap and effective-mass predictions; the model recovers cases metallic at PBE but insulating at HSE06.Source: §1 · p. 3
2 Results
O-kRystal uses lower-cost PBE band energies and structured orbital information to predict state-dependent corrections across crystal momentum and spin. It improves HSE06 band-energy predictions across diverse materials, generalizes beyond supervised k-points, and extends to correlated materials, derived properties, and G0W0 quasiparticle energies.
2.1 The O-kRystal Framework
- O-kRystal predicts target-theory energies by adding learned state- and momentum-dependent corrections to PBE energies, using real- and reciprocal-space orbital representations.Its periodic E(3)-equivariant graph neural network uses localized real-space interactions and reciprocal-space features to represent queried k-points.Source: §2.1 · p. 5
2.2 Target-theory band structure prediction
- 0.171 ± 0.009 eV is O-kRystal’s entry-balanced band-energy MAE within 5 eV of the band-edge midpoint, across three models and four energy windows.The benchmark uses 5,988 parent materials; O-kRystal has the lowest MAE among the compared methods in all four windows.Source: Figure 2 · p. 6, §2.2 · pp. 6–7
2.3 Generalization across the Brillouin Zone
- 0.121 eV is the median MAE at unseen path-interior points, comparable to 0.122 eV at supervised high-symmetry points; dense off-path predictions also improve DOS reconstruction over PBE.Across 356 insulating and 225 metallic materials, median DOS errors are 0.224 and 0.210, respectively, versus 0.874 and 0.457 for PBE.Source: §2.3 · pp. 7, 9, Figure 3 · p. 8
2.4 Electronic-character recovery and strongly correlated materials
- On six held-out strongly correlated materials, O-kRystal reduces dense-mesh band-energy MAE from 1.532 eV for PBE to 0.479 eV, a 68.7% reduction.For orbital-resolved errors, O-kRystal improves both transition-metal 3d and oxygen 2p channels versus PBE; its 3d improvement over Geometry GNN occurs in all six materials.Source: §2.4 · pp. 9–10, Figure 4 · p. 10
2.5 Derived electronic properties
- 0.203 eV is the MAE of O-kRystal-derived fundamental gaps against HSE06, compared with 0.926 eV for PBE; gap-informed masses improve experimental MAPE from 32.05% to 27.05%.Effective masses are reconstructed from predicted gaps and PBE curvature information, rather than directly predicted from HSE06 curvature.Source: §2.5 · pp. 11–12, Figure 5 · p. 11
2.6 Extension to G0W0 quasiparticle energies
- 0.126 ± 0.017 eV is the quasiparticle-state MAE on 286 two-dimensional semiconductors using a GPAW PBE baseline; the band-gap MAE is 0.185 ± 0.035 eV.Using independently calculated FHI-aims PBE bands instead increases the state MAE to 0.168 ± 0.010 eV and the gap MAE to 0.206 ± 0.027 eV.Source: Figure 6 · p. 12, §2.6 · p. 13
3 Discussion
O-kRystal uses lower-cost electronic structure as both an energy baseline and a representation of the electronic state, with orbital features adding information for material- and state-specific corrections. Its learned corrections generalize beyond supervised high-symmetry points, while the method remains limited to predicting state energies rather than target-theory wavefunctions.
3 Discussion
- Orbital features add information for material- and state-specific deviations, especially in correlation-sensitive states, while the full dual-space model achieves progressively lower errors than geometry-based and real-space alternatives.O-kRystal uses lower-cost electronic structure as both the Δ-learning baseline and a representation of the electronic state.Source: §3 · p. 13
- Accuracy remains essentially unchanged at unseen path-interior points, and dense Brillouin-zone predictions reconstruct target-level spectral quantities away from conventional band paths without target-theory calculations at each queried k-point.Source: §3 · pp. 13–14
- O-kRystal recovers metal-to-insulator changes and persistent metallicity, and resolves orbital-dependent placement of transition-metal d and ligand p states for screening candidates for more costly calculations.The discussion suggests screening for cases where DFT gives an inadequate low-energy electronic structure, including changes associated with doping, pressure, or strain.Source: §3 · p. 14
- The framework requires a converged baseline calculation and predicts state energies rather than wavefunctions, so its orbital-projected reconstructions retain baseline projections and cannot capture target-induced hybridization or wavefunction-character changes.Source: §3 · p. 14
4 Methods
The methods describe O-kRystal’s use of PBE band information and dual-space orbital features to learn state-resolved energy corrections, including its dataset, architecture, training objective, baselines, and evaluation procedures. They also define downstream protocols for gap-informed effective-mass reconstruction and for a two-dimensional G₀W₀ quasiparticle benchmark.
4.1 Dataset construction and electronic-structure data
- The primary dataset pairs PBE and HSE06 calculations for 5,988 three-dimensional materials, with parent-level splits and supervision at high-symmetry k-points.The 6,900 band-structure entries include separate spin channels; materials with PBE–HSE06 magnetic-moment differences above 0.5 µB are excluded. Errors are assessed in PBE-side energy windows around the band-edge midpoint.Source: §4.1 · pp. 14–15
4.2 Orbital representation
- O-kRystal represents the PBE electronic state using Hamiltonian, overlap, density, and spin-density orbital matrices in real and reciprocal space, with Bloch-phase conditioning.Real-space pair blocks form a periodic directed atom-image graph; reciprocal-space features are related by a Bloch transform and include k-dependent on-site blocks.Source: §4.2 · pp. 15–16
4.3 O-kRystal architecture
- The architecture combines an affine band-correction branch with a dual-space residual branch, using signed band queries and PBE-derived band descriptors.The residual is orthogonalized against its mean and linear component along PBE dispersion, so it captures remaining k-dependent corrections; the model predicts energies, not target wavefunctions or orbital projections.Source: §4.3 · pp. 16–17
4.4 Learning objective and optimization
- Training minimizes a weighted SmoothL1 state-energy correction loss, with additional penalties for bandwise offsets and errors in reconstructed fundamental gaps.The SmoothL1 transition parameter is 0.1 eV; models use AdamW for 100 epochs, and checkpoints are selected by validation MAE averaged across four energy windows.Source: §4.4 · pp. 18–19
4.5 Baseline models
- Comparisons include scissor and Linear+KRR corrections, a geometry-only GNN, and a Real+Bloch-phase ablation that omits reciprocal-space orbital matrices.Baselines use matched high-symmetry-point training and test states; the ablation matches the full model's training and evaluation settings.Source: §4.5 · pp. 19–20
4.6 Evaluation
- Evaluation measures matched-state band-energy MAE across energy windows, tests predictions at unseen path-interior k-points, and reconstructs DOS on dense Brillouin-zone meshes.DOS reconstruction uses tetrahedron integration and Gaussian broadening with σ = 0.10 eV; electronic-character classifications are compared with HSE06 references.Source: §4.6 · pp. 20–21
4.7 Downstream benchmarks
- A separate six-material correlated benchmark evaluates band-energy errors and transition-metal 3d and oxygen 2p projected density-of-states distributions against HSE06.The PDOS comparison holds PBE Mulliken orbital weights fixed across methods, isolating energy-placement differences rather than changes in wavefunctions or hybridization.Source: §4.7 · pp. 21–22
- The primary bulk experimental gap benchmark uses ten eligible materials, excluding monolayers, Pb chalcogenides, and black phosphorus for stated data or training-chemistry limitations.The benchmark uses mstar60; required PBE, HSE06, and experimental quantities must be available, and constituent elements must occur in the three-dimensional training set.Source: §4.7 · p. 22
- The effective-mass protocol reconstructs masses from PBE masses and a regularized O-kRystal-to-PBE band-gap ratio, with a mass-dependent exponent calibrated without the held-out chemical family.It evaluates 31 directional observations across ten materials using family-wise holdout; material-level percentage errors are averaged equally, and experimental masses are reserved for final evaluation.Source: §4.7 · p. 22
4.8 G0W0 quasiparticle benchmark
- The G0W0 benchmark comprises 286 nonmagnetic two-dimensional semiconductor records with quasiparticle energies for eight highest valence and four lowest conduction states, using GPAW PBE starting points.It compares GPAW and independently calculated FHI-aims PBE eigenvalue baselines while holding labels, k-point sets, and orbital inputs fixed; separate models are trained because the baseline changes the Δ-learning targets.Source: §4.8 · pp. 22–23
- The G0W0 task is trained separately with the same orbital-learning architecture and state-resolved Δ-learning, querying high-symmetry points and up to nine evenly spaced interior points per path segment.Five-fold cross-validation uses three folds for training, one for validation, and one held-out test fold; performance is reported as twelve-state MAE and fundamental-gap error, averaged across folds with sample-standard-deviation uncertainties.Source: §4.8 · p. 23
S1 Dataset construction and splits
The benchmark was built from paired PBE–HSE06 calculations, filtering out materials with large magnetic-moment discrepancies and excluding materials reserved for downstream benchmarks. The remaining parent materials were grouped into three train, validation, and test splits.
S1 Dataset construction and splits
- 5,988 parent materials and 6,900 band-structure entries remain after filtering magnetic-moment discrepancies above 0.5 µB and excluding materials reserved for downstream benchmarks.The preceding filter leaves 6,045 parents and 6,978 entries; spin-polarized parents contribute separate spin-channel entries.Source: S1 Dataset construction and splits · p. 29, Table S1 · p. 30
- The retained datasets contain 59 chemical elements, and spin-polarized parents are represented by separate spin-up and spin-down entries.The final dataset includes 933 spin-polarized parents represented by 1,866 explicit spin-channel entries.Source: S1 Dataset construction and splits · p. 29, Table S1 · p. 30
- Approximately 8:1:1 training, validation, and test partitions are grouped by parent material, with corresponding spin channels and repeated entries kept together.Three independent grouped splits are used for the reported model/split replicates.Source: S1 Dataset construction and splits · p. 29
S2 Orbital representation, architecture, and training
O-kRystal combines PBE orbital information in real and reciprocal space to predict state-dependent energy corrections. Its architecture pairs a real-space, Bloch-phase-conditioned affine branch with a reciprocal-space residual branch, trained on high-symmetry-point corrections.
Orbital representation
- The inputs include atom-centered Hamiltonian, overlap, density, and spin-density blocks, with real-space blocks describing local interactions and reciprocal-space matrices supplying k-resolved information.Non-spin-polarized entries use a zero spin-density channel; production inputs use real reciprocal components and disable imaginary channels.Source: S2 Orbital representation,… · p. 31
Architecture
- O-kRystal jointly trains a real-space, Bloch-phase-conditioned affine branch and a dual-space residual branch using explicit reciprocal-space orbital matrices.The residual is projected on the high-symmetry reference set to remove its constant and PBE-linear components before addition to the affine correction.Source: S2 Orbital representation,… · p. 31
State representation
- Signed band queries and five PBE-derived descriptors let the model accommodate variable numbers of valence and conduction states.The descriptors summarize band mean, standard deviation, minimum, maximum, and width relative to the band edge.Source: S2 Orbital representation,… · p. 31
Training
- Training targets state-resolved PBE-to-HSE06 corrections at stored high-symmetry k-points, using state, band-offset, and indirect-gap losses.The final objective is Lstate + Loffset + 0.2 Lgap; other auxiliary objectives have zero weight in the production model.Source: S2 Orbital representation,… · p. 31
Training and selection
- Checkpoints are selected by the lowest unweighted mean validation MAE across the 2, 5, 10, and 20 eV windows.Training uses 100 epochs, AdamW, a 20-epoch linear warm-up, and cosine learning-rate decay.Source: S2 Orbital representation,… · p. 31
Comparison models
- The comparison controls include scissor and Linear+KRR baselines, a Geometry GNN without orbital matrices, and Real+Bloch phase without explicit reciprocal-space matrices.The neural representation comparison uses matched seed-specific training subsets, the same objective and optimization schedule, and the same high-symmetry evaluation protocol.Source: S2 Orbital representation,… · p. 32, Table S3 · p. 34
S3 Contribution of reciprocal-space orbital information
The representation comparison finds that the full dual-space model’s error advantage becomes clearer with more training data, particularly over the broader energy window.
Comparison setup
- Geometry GNN removes orbital matrices, while Real+phase keeps real-space orbital blocks and Bloch-phase features but omits explicit reciprocal-space matrices.Geometry GNN uses a distance-selected graph, whereas Real+phase and O-kRystal share the electronically preselected real-space orbital graph.Source: S3 Contribution of reciprocal-space… · p. 35
Representation ablation
- The full dual-space model has the lowest error at the largest training size in both energy windows, with a clearer separation over 20 eV.All three representations perform similarly in the smallest-data regime; real-space orbital information improves on the geometry-based representation.Source: Figure S1 · p. 35, S3 Contribution of reciprocal-space… · p. 35
S4 Reciprocal-space generalization
O-kRystal is trained on stored high-symmetry points and evaluated at unseen points along the corresponding paths. Interior-point errors are nearly unchanged in aggregate, with location-related patterns differing between insulating and metallic systems.
Evaluation
- The unseen-point evaluation uses complete query sets for each path and the same state-matching and energy-window conventions as the primary benchmark.O-kRystal is trained only on stored high-symmetry k-points for this task.Source: S4 Reciprocal-space generalization · p. 36
Unseen-point diagnostics
- The interior-point versus high-symmetry-point MAE difference is centered near zero, while insulating errors rise gradually with distance from supervised points and metallic fluctuations reflect a few difficult materials.For insulators, error is nearly independent of normalized path position; the high-error metallic bins are dominated primarily by Ti2P and Zr2P rather than a uniform trend.Source: S4 Reciprocal-space generalization · p. 36, Figure S2 · p. 37
Representative predictions
- The supplementary gallery shows full-path PBE, HSE06, and O-kRystal predictions for insulating, metallic, and spin-polarized examples, including GaO2, WC, and AFM-II MnO.The gallery includes examples of different PBE-to-HSE06 electronic-character changes and spin configurations.Source: Figure S3 · p. 38
S5 Strongly correlated materials
On a separate six-material correlated benchmark, O-kRystal reduces band-energy error relative to PBE and Geometry GNN across the evaluated energy windows. The benchmark also evaluates gap recovery and orbital-resolved spectral reconstruction using fixed PBE projection weights.
Benchmark construction
- The benchmark comprises six literature-supported correlated systems, with candidates excluded when calculations do not defensibly represent the supported phase or electronic character.The six retained materials are CrO2, LaCoO3, LaNiO3, AFM-II MnO, AFM-II NiO, and YTiO3.Source: S5 Strongly correlated materials · p. 39
Band-energy benchmark
- 0.479 eV is O-kRystal’s equal-material mean band-energy MAE within 5 eV, versus 1.532 eV for PBE and 0.692 eV for Geometry GNN.Over 20 eV, the corresponding MAEs are 0.410, 1.472, and 0.581 eV; O-kRystal has the lowest 5 eV error for all six materials.Source: Table S4 · p. 39, S5 Strongly correlated materials · p. 39
Fundamental-gap recovery
- O-kRystal’s fundamental-gap MAE is 1.537 eV, versus 1.953 eV for PBE and 1.769 eV for Geometry GNN, and it recovers HSE06 character for five of six materials.CrO2 is the remaining O-kRystal classification failure: HSE06 is metallic, while the prediction is insulating.Source: S5 Strongly correlated materials · pp. 39–40, Table S5 · p. 40
Orbital-resolved PDOS
- For transition-metal 3d states, O-kRystal lowers normalized PDOS error relative to Geometry GNN across all six materials, reducing the equal-material mean from 0.989 to 0.709.For oxygen 2p states, aggregate error falls from 0.753 to 0.631, but material-level gains are less uniform; all spectra use identical PBE Mulliken weights.Source: Table S6 · p. 40, S5 Strongly correlated materials · pp. 40–41
Direct-source MnO case study
- A separate AFM-II MnO case study reconstructs the spectrum using PBE Mulliken weights and O-kRystal-predicted HSE06 energies on a shared 12 × 12 × 12 mesh.This illustrative spectrum is kept numerically separate from the rebuilt six-material PDOS aggregate.Source: S5 Strongly correlated materials · p. 41
S6 Experimental gap and effective-mass benchmark
The bulk effective-mass benchmark evaluates a gap-informed reconstruction on 31 observations across ten materials. It rescales PBE effective masses using predicted and PBE fundamental gaps, with calibration separated from experimental evaluation.
S6 Experimental gap and effective-mass benchmark
- Effective masses are reconstructed by rescaling PBE effective masses according to the ratio of O-kRystal-predicted and PBE fundamental gaps.The gap–mass relation is a calibrated generalization motivated by the Kane model, not a direct Kane Hamiltonian implementation.Source: S6 Experimental gap and effective-mass… · p. 42
- Calibration uses HSE06 effective masses from the remaining material families, while experimental masses are reserved for final evaluation.Gaps are evaluated over the complete stored band path, and parameters are calibrated separately for each held-out family.Source: S6 Experimental gap and effective-mass… · p. 42
- Grid search selects family-specific reconstruction parameters by minimizing material-balanced mean absolute logarithmic error against calibration-family HSE06 masses.The reported parameter pairs are (4.4, 0.50) for III–V, (4.0, 0.25) for chalcogenides, and (2.9, 0.50) for other semiconductors.Source: S6 Experimental gap and effective-mass… · p. 42
- 31 directional effective-mass observations across ten bulk materials make up the primary benchmark.The cohort excludes monolayers, out-of-domain Pb chalcogenides, and cases missing reliable required quantities.Source: S6 Experimental gap and effective-mass… · p. 42, Table S7 · p. 43
- The reported effective-mass values are benchmark observations, and O-kRystal* denotes a family-held-out gap-informed reconstruction rather than a direct model output.The cohort and observation listings are provided in Supplementary Tables S7 and S8.Source: S6 Experimental gap and effective-mass… · p. 42, Table S8 · p. 43
S7 G0W0 quasiparticle benchmark
The G0W0 benchmark evaluates state-resolved quasiparticle corrections for 286 two-dimensional nonmagnetic semiconductors. Its matched model variants differ only in the PBE eigenvalue baseline used to define correction targets.
S7 G0W0 quasiparticle benchmark
- The benchmark covers 286 two-dimensional nonmagnetic semiconductors, with eight valence and four conduction states per material, targeting one-shot G0W0 spectra from a GPAW PBE starting point.Training and validation use high-symmetry vertices and up to nine evenly spaced stored interior points per path segment.Source: S7 G0W0 quasiparticle benchmark · p. 44
- Five-fold material-level cross-validation evaluates state and band-gap MAEs on held-out materials, with a separate fold for validation checkpoint selection.The two independently trained variants otherwise share labels, k-point sets, orbital inputs, model settings, sampling, and optimization.Source: S7 G0W0 quasiparticle benchmark · p. 44
- Supplementary Figure S4 reports state-resolved errors across VB8–VB1 and CB1–CB4 for the two baseline variants.Supplementary Table S9 reports fold-resolved state and band-gap MAEs in eV.Source: Figure S4 · p. 44, Table S9 · p. 45
S8 Computational and reproducibility details
The implementation details report training cost, model size, reproducibility artifacts, and cumulative computational accounting. The HSE06-to-PBE resource ratio is reported as cumulative CPU usage, not an end-to-end speedup.
S8 Computational and reproducibility details
- The final three-dimensional O-kRystal model takes approximately 10 GPU-hours to train for 100 epochs on a single NVIDIA H200 GPU.Training takes approximately 6 minutes per epoch; batches contain 1–12 entries subject to a 700-atom limit, and evaluation uses the raw best_avg_win checkpoint.Source: S8 Computational and reproducibility… · p. 46
- The seed-42 model has 8,940,709 trainable parameters, with checkpoint, configuration, split manifests, UID lists, and evaluation materials retained for reproducibility.The source code, configurations, evaluation scripts, split manifests, and numerical figure-source data are slated for release with the public repository.Source: S8 Computational and reproducibility… · p. 46
- For 6,576 source materials, cumulative CPU use is 88,051 core-hours for PBE and 1,600,661 core-hours for HSE06, an aggregate HSE06-to-PBE ratio of 18.18.These cumulative resource totals are not wall-clock times and are not used to infer an end-to-end speedup.Source: S8 Computational and reproducibility… · p. 46