Source-linked AI summary
General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian
Xiaoxun Gong, He Li, Nianlong Zou, Runzhang Xu, Wenhui Duan, Yong Xu
TL;DR
Designing neural networks that preserve physical symmetries remains a challenge for DFT Hamiltonian learning. DeepH-E3 addresses this with an E(3)-equivariant representation that handles spin-orbit coupling and enables accurate, efficient calculations for large materials. Its remaining bottleneck is diagonalizing increasingly large Hamiltonian matrices.
Problem
Existing DeepH approaches required local-coordinate transformations to learn covariant Hamiltonians, increasing computational load, while incorporating equivariant neural networks remained elusive, especially with spin-orbit coupling.
Method
DeepH-E3 uses an E(3)-equivariant message-passing neural network to map atomic structures to DFT Hamiltonians while handling spin-orbital representations and complex-valued outputs.
Results
DeepH-E3 achieves sub-meV DFT Hamiltonian prediction accuracy and reduces computational costs by several orders of magnitude for large-scale materials.
Takeaways & Limitations
DeepH-E3 enables accurate and efficient electronic-structure calculations for large material systems and supports applications such as Moiré-twisted material databases.
Takeaways & Limitations
For increasingly large structures, diagonalizing the Hamiltonian to obtain wavefunction-related quantities becomes the eventual bottleneck.
Abstract
from arXiv · showhide
Combination of deep learning and ab initio calculation has shown great promise in revolutionizing future scientific research, but how to design neural network models incorporating a priori knowledge and symmetry requirements is a key challenging subject. Here we propose an E(3)-equivariant deep-learning framework to represent density functional theory (DFT) Hamiltonian as a function of material structure, which can naturally preserve the Euclidean symmetry even in the presence of spin-orbit coupling. Our DeepH-E3 method enables very efficient electronic-structure calculation at ab initio accuracy by learning from DFT data of small-sized structures, making routine study of large-scale supercells ($> 10^4$ atoms) feasible. Remarkably, the method can reach sub-meV prediction accuracy at high training efficiency, showing state-of-the-art performance in our experiments. The work is not only of general significance to deep-learning method development, but also creates new opportunities for materials research, such as building Moiré-twisted material database.
I. INTRODUCTION
Deep learning can address the accuracy–efficiency challenge in ab initio materials calculations, but Hamiltonian prediction must preserve covariance under rotations and gauge transformations. DeepH-E3 introduces an E(3)-equivariant framework that incorporates these requirements, including spin and spin-orbit coupling.
- Deep-learning models have enabled efficient prediction of material energies, forces, and other properties, addressing the accuracy–efficiency dilemma of ab initio calculations.
- Hamiltonian matrix elements must transform covariantly under rotations or gauge transformations, so this symmetry should be built into the neural network.
- Local-coordinate transformations can convert the covariance problem into an invariant one, but they increase computational load and make performance sensitive to coordinate selection.
- With spin-orbit coupling, spin and orbital degrees of freedom are coupled and must transform together under global rotations.
- DeepH-E3 represents the spin-orbital DFT Hamiltonian as a function of atomic structure using an E(3)-equivariant neural network.The framework is presented as a way to incorporate covariance requirements while accounting for electronic spin and spin-orbit coupling.
II. REALIZATION OF EQUIVARIANCE
The method constructs an E(3)-equivariant mapping from atomic structures to DFT Hamiltonians using symmetry-aware features and Wigner–Eckart transformations. It handles spin-orbit coupling by restricting half-integer representations to the output and combining real network outputs into complex spin-orbital quantities.
- The structure-to-Hamiltonian mapping must transform covariantly when atomic coordinates change between rotated reference frames.Translation symmetry is handled through relative positions, while rotations require nontrivial transformations of Hamiltonian matrices.
- The equivariant neural network propagates features carrying SO(3) irreducible representations, preserving the transformation rule through its internal and output features.
- Atomic numbers and interatomic distances form scalar inputs, while spherical harmonics of relative-position directions provide higher-angular-momentum input vectors.
- The Wigner–Eckart layer combines output representations using tensor-product decompositions so the resulting tensors represent equivariant Hamiltonian blocks.The tensor-product rule decomposes l1 ⊗ l2 into representations from |l1 − l2| through l1 + l2.
- Spin-orbit coupling complicates the construction because spin-1/2 requires complex-valued half-integer representations and changes sign under a 2π rotation.
- Half-integer representations can be confined to the final output layer, where integer representations are converted into the required spin-orbital form.
- For SOC, two real neural-network outputs are combined into complex vectors before the Wigner–Eckart layer constructs the spin-orbital Hamiltonian.A change of basis handles complex-conjugate representations, including for half-integer angular momentum.
III. NEURAL NETWORK ARCHITECTURE OF DeepH-E3
DeepH-E3 uses an E(3)-equivariant message-passing architecture that propagates directional and neighborhood information while preserving equivariance. Tensor products, equivariant layers, nonlinear gates, and normalization transform learned edge features into Hamiltonian matrix blocks.
- The material structure is represented as a graph whose vertices are atoms, with edges for nonzero inter-site hopping and self-loops for intrasite coupling.
- Neighboring messages iteratively update vertex and edge features, and the final edge feature is linearly transformed into the Hamiltonian block H_ij.The output depends only on the neighborhood environment, supporting linear-scaling calculations and generalization through nearsightedness.
- Tensor-Field Network building blocks use representation-order and channel-indexed feature vectors, while E3Linear layers connect only equivalent representations.Gate layers use activated scalar features to scale non-scalar vector norms, preserving equivariant nonlinear processing.
- E3LayerNorm normalizes feature vectors using layer statistics while preserving equivariance, with analogous normalization for vertex and edge updates.The normalization includes numerical-stability and learnable affine parameters.
- Tensor product layers couple features with different angular orders using Clebsch-Gordan coefficients and learnable weights.This operation is abbreviated as z = (Ux) ⊗ (Vy).
- Equivariant convolution combines vertex and edge features with spherical harmonics to inject directional structural information into message passing.
IV. CAPABILITY OF DeepH-E3
DeepH-E3 incorporates Euclidean symmetry into a message-passing framework for learning DFT Hamiltonians from small structures and applying them to large, twisted and spin–orbit-coupled materials. It achieves sub-meV accuracy while substantially reducing errors and computational cost in large-scale electronic-structure calculations.
- Method: DeepH-E3 incorporates global Euclidean symmetry into message passing to represent spin-orbital DFT Hamiltonians and support efficient, accurate prediction.The method learns from small structures and generalizes to different structure sizes without further DFT calculations.
- Method: The architecture updates vertex and edge features through equivariant convolutions that encode interatomic distances and directional information.Elemental embeddings initialize vertices, Gaussian expansions initialize edges, and repeated update blocks propagate these features.
- Twisted bilayer graphene: DeepH-E3 reduces averaged Hamiltonian-element MAEs by more than a factor of 2 relative to DeepH, reaching 0.2–0.3 meV for twisted bilayer graphene.The MAEs gradually decrease with increasing Moiré supercell size, indicating generalization across twist angles and system sizes.
- Twisted bilayer graphene: For magic-angle TBG with 11,164 atoms per supercell, DeepH-E3 reproduces the flat bands near the Fermi level in agreement with DFT and a continuum model.Minor discrepancies away from the Fermi level are partly attributed to differences in basis and pseudopotential between benchmark calculations.
- Efficiency: DeepH-E3 reduces the computational cost of large-material studies by several orders of magnitude, requiring tens of GPU hours for training and minutes for magic-angle-TBG inference.The training is performed once, after which Hamiltonian construction scales linearly with Moiré supercell size.
- Spin–orbit coupling: For twisted Bi2Te3, DeepH-E3 predicts DFT Hamiltonians with and without strong spin–orbit coupling and captures a SOC-driven transition from Z2 = 0 to Z2 = 1.The transition is identified through closing and reopening of the band gap at the Γ point.
V. DISCUSSION
DeepH-E3 explicitly preserves Hamiltonian covariance with SOC and supports accurate, efficient electronic-structure calculations for large materials. Its main remaining bottleneck is diagonalizing increasingly large Hamiltonian matrices.
- V. DISCUSSION: DeepH-E3 represents the DFT Hamiltonian as an explicitly E(3)-equivariant function of atomic structure, including spin-orbit coupling.The framework incorporates covariance requirements into neural-network models and provides a code implementation.
- V. DISCUSSION: The method enables accurate and efficient electronic-structure calculations for large-scale materials beyond traditional ab initio approaches.The authors identify applications to rich physics and novel material properties at lower computational cost.
- V. DISCUSSION: For sufficiently large structures, Hamiltonian diagonalization becomes the bottleneck for obtaining wavefunction-related physical quantities.The authors distinguish this scaling difficulty from a limitation of DeepH-E3 itself and point to sparse-matrix O(N) algorithms as a future direction.
- V. DISCUSSION: DeepH-E3 could support high-throughput databases of Moiré-twisted materials, including structures with incommensurate twist angles.One trained model can be used for all twisted structures of a material, and the method does not rely on periodic boundary conditions.
Appendix A: Datasets
The appendix describes graphene and MoS2 datasets generated from ab initio molecular dynamics, with specified sampling, splits, electronic-structure codes, and localized orbital bases.
- Appendix A: Datasets: The graphene dataset contains 450 structures from 6×6 supercells, split into 270 training, 90 validation, and 90 testing structures.Frames were sampled from 5000 molecular-dynamics frames at 300K using a 1fs timestep.
- Appendix A: Datasets: The MoS2 dataset contains 500 structures from 5×5 supercells, split into 300 training, 100 validation, and 100 testing structures.The first 500 of 1000 molecular-dynamics frames were discarded as unequilibrated before sampling.
- Appendix A: Datasets: Graphene and MoS2 Hamiltonians were computed with different electronic-structure setups: VASP for dataset generation and OpenMX for MoS2 Hamiltonians.The passage specifies PBE exchange-correlation functionals and the relevant pseudopotential approaches.
Appendix B: Details of neural network models
The appendix specifies training objectives, basis choices, activation functions, and the E3NN-based architecture used for DeepH-E3 models.
- Appendix B: Details of neural network models: All neural networks are trained by minimizing mean-squared error against DFT Hamiltonian matrices, while reported MAEs compare predictions with DFT results.Material physical quantities are derived from the output Hamiltonian matrix.
- Appendix B: Details of neural network models: The model uses a Gaussian basis adopted from prior work.The supplied passage introduces the basis but does not provide its full defining expression.
- Appendix B: Details of neural network models: The gate layer uses SiLU and Sigmoid activation functions for its two activation components.The appendix identifies these choices as φ1=SiLU and φ2=Sigmoid.
- Appendix B: Details of neural network models: The ENN uses three vertex-update blocks and three edge-update blocks implemented with e3nn 0.3.5 and PyTorch 1.9.0.Initial edge and vertex features are 64 × 0e, with higher-order equivariant representations used internally.